[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:40 dist-test-slave-f7th krb5kdc[22589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:40 dist-test-slave-f7th krb5kdc[22589](info): set up 2 sockets
Aug 10 06:49:40 dist-test-slave-f7th krb5kdc[22589](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 06:49:41.677554 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.461s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:49:41 dist-test-slave-f7th krb5kdc[22589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:49:41.858965 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:33543
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:33543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:42.404870 22611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:42.405309 22611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:42.405507 22611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:42.414501 22611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:42.414636 22611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:42.414736 22611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:42.414880 22611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:42.414978 22611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:42.427834 22611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:33543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:33543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:42.429854 22611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:42.431809 22611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:42.450551 22617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:42.450274 22624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:42.450400 22618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:42.453500 22611 server_base.cc:1034] running on GCE node
I20260810 06:49:42.454551 22611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:42.456840 22611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:42.458084 22611 hybrid_clock.cc:648] HybridClock initialized: now 1786344582457930 us; error 97 us; skew 500 ppm
I20260810 06:49:42.458596 22611 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:42 dist-test-slave-f7th krb5kdc[22589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344582, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:42.462908 22611 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:49:42.465847 22611 webserver.cc:460] Webserver started at http://127.22.11.190:34023/ using document root <none> and password file <none>
I20260810 06:49:42.466987 22611 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:42.467162 22611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:42.467559 22611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:42.472078 22611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "e77fe1203b9d4fada42f08baa8971f00"
format_stamp: "Formatted at 2026-08-10 06:49:42 on dist-test-slave-f7th"
server_key: "db2e0eabe713568d83e982875e631c6f"
server_key_iv: "0ad121406ec9036ae8cccfdc288579f5"
server_key_version: "encryptionkey@0"
I20260810 06:49:42.472857 22611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "e77fe1203b9d4fada42f08baa8971f00"
format_stamp: "Formatted at 2026-08-10 06:49:42 on dist-test-slave-f7th"
server_key: "db2e0eabe713568d83e982875e631c6f"
server_key_iv: "0ad121406ec9036ae8cccfdc288579f5"
server_key_version: "encryptionkey@0"
I20260810 06:49:42.481422 22611 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20260810 06:49:42.487329 22634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:42.489195 22611 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:49:42.489405 22611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "e77fe1203b9d4fada42f08baa8971f00"
format_stamp: "Formatted at 2026-08-10 06:49:42 on dist-test-slave-f7th"
server_key: "db2e0eabe713568d83e982875e631c6f"
server_key_iv: "0ad121406ec9036ae8cccfdc288579f5"
server_key_version: "encryptionkey@0"
I20260810 06:49:42.489580 22611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:42.504171 22611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:42.509833 22611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:42.510233 22611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:42.547124 22611 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:33543
I20260810 06:49:42.547132 22724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:33543 every 8 connection(s)
I20260810 06:49:42.549675 22611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:42.557473 22725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:42.557665 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 22611
I20260810 06:49:42.558024 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:49:42.560251 22574 external_mini_cluster.cc:1428] Setting key f1042481cd397ca7a9c3a8ad74493645
I20260810 06:49:42.574959 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00: Bootstrap starting.
I20260810 06:49:42.581092 22725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:42.583902 22725 log.cc:824] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:42.588652 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00: No bootstrap required, opened a new log
Aug 10 06:49:42 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:42.600037 22725 raft_consensus.cc:348] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77fe1203b9d4fada42f08baa8971f00" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33543 } }
I20260810 06:49:42.600562 22725 raft_consensus.cc:374] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:42.600854 22725 raft_consensus.cc:729] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e77fe1203b9d4fada42f08baa8971f00, State: Initialized, Role: FOLLOWER
I20260810 06:49:42.602139 22725 consensus_queue.cc:260] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77fe1203b9d4fada42f08baa8971f00" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33543 } }
I20260810 06:49:42.602705 22725 raft_consensus.cc:388] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:42.603040 22725 raft_consensus.cc:482] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:42.603327 22725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:42.606668 22725 raft_consensus.cc:504] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77fe1203b9d4fada42f08baa8971f00" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33543 } }
I20260810 06:49:42.607685 22725 leader_election.cc:302] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e77fe1203b9d4fada42f08baa8971f00; no voters:
I20260810 06:49:42.608880 22725 leader_election.cc:288] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:42.609093 22736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:42.611342 22736 raft_consensus.cc:686] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [term 1 LEADER]: Becoming Leader. State: Replica: e77fe1203b9d4fada42f08baa8971f00, State: Running, Role: LEADER
I20260810 06:49:42.612584 22736 consensus_queue.cc:237] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77fe1203b9d4fada42f08baa8971f00" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33543 } }
I20260810 06:49:42.613111 22725 sys_catalog.cc:564] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:42.614238 22730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:42.566275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.22.11.190:33543)
0810 06:49:42.567961 (+ 1686us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:42.567980 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:42.569696 (+ 1716us) server_negotiation.cc:407] Connection header received
0810 06:49:42.582912 (+ 13216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:42.582942 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:42.583676 (+ 734us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:42.584186 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:42.587965 (+ 3779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:42.590671 (+ 2706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:42.592693 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:42.593961 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:42.594213 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:42.601670 (+ 7457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:42.601745 (+ 75us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:42.601792 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:42.601842 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:42.606372 (+ 4530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:42.607347 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:42.607353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:42.607359 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:42.607651 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:42.608254 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:42.608259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:42.608262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:42.609126 (+ 864us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:42.609771 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:42.611399 (+ 1628us) server_negotiation.cc:299] Negotiation successful
0810 06:49:42.611919 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1171,"thread_start_us":668,"threads_started":1}
I20260810 06:49:42.620922 22737 sys_catalog.cc:455] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e77fe1203b9d4fada42f08baa8971f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77fe1203b9d4fada42f08baa8971f00" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33543 } } }
I20260810 06:49:42.621438 22737 sys_catalog.cc:458] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:42.621759 22741 sys_catalog.cc:455] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e77fe1203b9d4fada42f08baa8971f00. Latest consensus state: current_term: 1 leader_uuid: "e77fe1203b9d4fada42f08baa8971f00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e77fe1203b9d4fada42f08baa8971f00" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33543 } } }
I20260810 06:49:42.622061 22741 sys_catalog.cc:458] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:42.624878 22751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:42.641438 22751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:42.657171 22751 catalog_manager.cc:1347] Generated new cluster ID: c3c4240124274d758f60f2fa98e3f545
I20260810 06:49:42.657368 22751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:42.669422 22751 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:42.671682 22751 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:42.680709 22751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e77fe1203b9d4fada42f08baa8971f00: Generated new TSK 0
I20260810 06:49:42.681860 22751 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:49:42.749575 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:33543
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:43.278517 22772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:43.278857 22772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:43.279011 22772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:43.288594 22772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:43.288784 22772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:43.288946 22772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:49:43.302063 22772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:33543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:43.303455 22772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:43.304905 22772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:43.320144 22779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.320840 22780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:43.320938 22782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:43.321640 22772 server_base.cc:1034] running on GCE node
I20260810 06:49:43.322304 22772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:43.323248 22772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:43.324482 22772 hybrid_clock.cc:648] HybridClock initialized: now 1786344583324400 us; error 36 us; skew 500 ppm
I20260810 06:49:43.324828 22772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:43 dist-test-slave-f7th krb5kdc[22589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:43.328650 22772 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:49:43.330097 22772 webserver.cc:460] Webserver started at http://127.22.11.129:36361/ using document root <none> and password file <none>
I20260810 06:49:43.330871 22772 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:43.330987 22772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:43.331290 22772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:43.334733 22772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "c1e29a06a7734fdf95c866d795ddaed3"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-f7th"
server_key: "3285a27d9cdb7166a19e5af8345629c8"
server_key_iv: "b00872e6be56a25ac94b6b0be23438da"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.335564 22772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "c1e29a06a7734fdf95c866d795ddaed3"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-f7th"
server_key: "3285a27d9cdb7166a19e5af8345629c8"
server_key_iv: "b00872e6be56a25ac94b6b0be23438da"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.341677 22772 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260810 06:49:43.346022 22791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:43.347327 22772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 06:49:43.347520 22772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "c1e29a06a7734fdf95c866d795ddaed3"
format_stamp: "Formatted at 2026-08-10 06:49:43 on dist-test-slave-f7th"
server_key: "3285a27d9cdb7166a19e5af8345629c8"
server_key_iv: "b00872e6be56a25ac94b6b0be23438da"
server_key_version: "encryptionkey@0"
I20260810 06:49:43.347774 22772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:43.362774 22772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:43.367643 22772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:43.368001 22772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:43.369248 22772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:43.371122 22772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:43.371228 22772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:43.371428 22772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:43.371555 22772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:43.433403 22772 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:46079
I20260810 06:49:43.433485 22994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:46079 every 8 connection(s)
I20260810 06:49:43.434940 22772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:43.439985 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 22772
I20260810 06:49:43.440254 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:43.441123 22574 external_mini_cluster.cc:1428] Setting key 18af8857b6f15b4c8bb470d21e7c03e2
Aug 10 06:49:43 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344583, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:43.459810 22997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.438620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:48949 (local address 127.22.11.190:33543)
0810 06:49:43.439264 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:43.439269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:43.440774 (+ 1505us) server_negotiation.cc:407] Connection header received
0810 06:49:43.443425 (+ 2651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:43.443431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:43.443516 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:43.443637 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:43.445490 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.446713 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.448595 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.449185 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.449321 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.453261 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:43.453266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:43.453270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:43.453302 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:43.456040 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.456994 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.456998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.457000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.457182 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:43.457833 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:43.457838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:43.457841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:43.458188 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:43.458349 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:43.459258 (+ 909us) server_negotiation.cc:299] Negotiation successful
0810 06:49:43.459462 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":384,"threads_started":1}
I20260810 06:49:43.461292 22999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:43.439248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:33543 (local address 127.22.11.129:48949)
0810 06:49:43.440574 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:43.440640 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:43.443100 (+ 2460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:43.443813 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:43.443820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:43.444479 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:43.445323 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.445343 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.446940 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.446944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.448434 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:43.448449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:43.449461 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:43.449467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:43.449724 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:43.450458 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:43.450481 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:43.453036 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:43.456181 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.456189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.456205 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.456873 (+ 668us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.457299 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:43.457304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:43.457308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:43.457698 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:43.458319 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:43.458327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:43.458806 (+ 479us) client_negotiation.cc:769] Sending connection context
0810 06:49:43.459189 (+ 383us) client_negotiation.cc:240] Negotiation successful
0810 06:49:43.459817 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"thread_start_us":679,"threads_started":1}
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
I20260810 06:49:43.463006 22995 heartbeater.cc:347] Connected to a master server at 127.22.11.190:33543
I20260810 06:49:43.463521 22995 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:43.464541 22995 heartbeater.cc:511] Master 127.22.11.190:33543 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
I20260810 06:49:43.468801 22663 ts_manager.cc:194] Registered new tserver with Master: c1e29a06a7734fdf95c866d795ddaed3 (127.22.11.129:46079)
I20260810 06:49:43.471868 22663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:48949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:49:43.491328 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:33543
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:44.033532 23005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:44.033882 23005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:44.034081 23005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:44.043484 23005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:44.043660 23005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:44.043855 23005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:49:44.056327 23005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:33543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:44.057636 23005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:44.059288 23005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:44.076439 23017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:44.076823 23019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:44.077112 23021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:44.078513 23005 server_base.cc:1034] running on GCE node
I20260810 06:49:44.079236 23005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:44.080447 23005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:44.081660 23005 hybrid_clock.cc:648] HybridClock initialized: now 1786344584081599 us; error 50 us; skew 500 ppm
I20260810 06:49:44.082067 23005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:44 dist-test-slave-f7th krb5kdc[22589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344584, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:44.089829 23005 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:49:44.091671 23005 webserver.cc:460] Webserver started at http://127.22.11.130:45931/ using document root <none> and password file <none>
I20260810 06:49:44.092438 23005 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:44.092698 23005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:44.093068 23005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:44.096426 23005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "43582793d09544f5b40635c57f8503b8"
format_stamp: "Formatted at 2026-08-10 06:49:44 on dist-test-slave-f7th"
server_key: "12a75f9ebd8c6a1211e537bf1e2c3407"
server_key_iv: "9a1b31c4967bb50644a143b869fc7184"
server_key_version: "encryptionkey@0"
I20260810 06:49:44.097338 23005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "43582793d09544f5b40635c57f8503b8"
format_stamp: "Formatted at 2026-08-10 06:49:44 on dist-test-slave-f7th"
server_key: "12a75f9ebd8c6a1211e537bf1e2c3407"
server_key_iv: "9a1b31c4967bb50644a143b869fc7184"
server_key_version: "encryptionkey@0"
I20260810 06:49:44.103240 23005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 06:49:44.108042 23032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:44.109513 23005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:49:44.109717 23005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "43582793d09544f5b40635c57f8503b8"
format_stamp: "Formatted at 2026-08-10 06:49:44 on dist-test-slave-f7th"
server_key: "12a75f9ebd8c6a1211e537bf1e2c3407"
server_key_iv: "9a1b31c4967bb50644a143b869fc7184"
server_key_version: "encryptionkey@0"
I20260810 06:49:44.109939 23005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:44.139533 23005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:44.145191 23005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:44.145563 23005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:44.147056 23005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:44.149097 23005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:44.149232 23005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:44.149431 23005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:44.149510 23005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:44.225073 23005 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:36607
I20260810 06:49:44.225167 23221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:36607 every 8 connection(s)
I20260810 06:49:44.226645 23005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:44.230156 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23005
I20260810 06:49:44.230365 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:44.230819 22574 external_mini_cluster.cc:1428] Setting key 388d75b497a640383bcf1d9534061e2d
Aug 10 06:49:44 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344584, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:44.256465 23226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:44.230120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:53677 (local address 127.22.11.190:33543)
0810 06:49:44.233338 (+ 3218us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:44.233343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:44.233361 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:49:44.235933 (+ 2572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:44.235938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:44.236004 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:44.236098 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:44.238909 (+ 2811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.239723 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.242966 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.243635 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.243741 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:44.248650 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:44.248658 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:44.248663 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:44.248703 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:44.252923 (+ 4220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:44.253730 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:44.253736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:44.253739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:44.253913 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:44.254435 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:44.254440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:44.254442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:44.254796 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:44.254956 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:44.255750 (+ 794us) server_negotiation.cc:299] Negotiation successful
0810 06:49:44.255989 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3087,"thread_start_us":2955,"threads_started":1}
I20260810 06:49:44.257576 23227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:44.230872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:33543 (local address 127.22.11.130:53677)
0810 06:49:44.232390 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:44.232452 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:44.235623 (+ 3171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:44.236410 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:44.236424 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:44.237605 (+ 1181us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:44.238712 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:44.238740 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.239908 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.239913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:44.242813 (+ 2900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:44.242829 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:44.243959 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:44.243968 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:44.244345 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:44.245228 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:44.245273 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:44.248314 (+ 3041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:44.253044 (+ 4730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:44.253050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:44.253061 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:44.253600 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:44.254021 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:44.254024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:44.254026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:44.254310 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:44.254914 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:44.254919 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:44.255288 (+ 369us) client_negotiation.cc:769] Sending connection context
0810 06:49:44.255570 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 06:49:44.256052 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"thread_start_us":817,"threads_started":1}
I20260810 06:49:44.258957 23223 heartbeater.cc:347] Connected to a master server at 127.22.11.190:33543
I20260810 06:49:44.259332 23223 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:44.260557 23223 heartbeater.cc:511] Master 127.22.11.190:33543 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
I20260810 06:49:44.262391 22663 ts_manager.cc:194] Registered new tserver with Master: 43582793d09544f5b40635c57f8503b8 (127.22.11.130:36607)
I20260810 06:49:44.263458 22663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:53677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:49:44.289081 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:33543
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:49:44.475862 22995 heartbeater.cc:503] Master 127.22.11.190:33543 was elected leader, sending a full tablet report...
W20260810 06:49:44.873512 23232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:44.873878 23232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:44.874058 23232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:44.883721 23232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:44.883944 23232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:44.884193 23232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:49:44.897004 23232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:33543
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:44.898456 23232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:44.899947 23232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:44.921036 23242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:44.921145 23246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:44.922283 23239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:44.923349 23232 server_base.cc:1034] running on GCE node
I20260810 06:49:44.923964 23232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:44.925125 23232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:44.926461 23232 hybrid_clock.cc:648] HybridClock initialized: now 1786344584926297 us; error 71 us; skew 500 ppm
I20260810 06:49:44.926949 23232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:44 dist-test-slave-f7th krb5kdc[22589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344584, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:44.931437 23232 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:49:44.933382 23232 webserver.cc:460] Webserver started at http://127.22.11.131:45829/ using document root <none> and password file <none>
I20260810 06:49:44.934257 23232 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:44.934510 23232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:44.934947 23232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:44.938403 23232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "11d20185c9864d2e9c0321dc28dd95b2"
format_stamp: "Formatted at 2026-08-10 06:49:44 on dist-test-slave-f7th"
server_key: "873e26544859978d966cfc34d4f2fa25"
server_key_iv: "a9d390491b29d6f4eee5f4c26a3a6a1a"
server_key_version: "encryptionkey@0"
I20260810 06:49:44.939288 23232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "11d20185c9864d2e9c0321dc28dd95b2"
format_stamp: "Formatted at 2026-08-10 06:49:44 on dist-test-slave-f7th"
server_key: "873e26544859978d966cfc34d4f2fa25"
server_key_iv: "a9d390491b29d6f4eee5f4c26a3a6a1a"
server_key_version: "encryptionkey@0"
I20260810 06:49:44.946296 23232 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260810 06:49:44.951094 23258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:44.953040 23232 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 06:49:44.953307 23232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "11d20185c9864d2e9c0321dc28dd95b2"
format_stamp: "Formatted at 2026-08-10 06:49:44 on dist-test-slave-f7th"
server_key: "873e26544859978d966cfc34d4f2fa25"
server_key_iv: "a9d390491b29d6f4eee5f4c26a3a6a1a"
server_key_version: "encryptionkey@0"
I20260810 06:49:44.953570 23232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:44.974500 23232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:44.982133 23232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:44.982511 23232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:44.984432 23232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:44.987154 23232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:44.987290 23232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:44.987504 23232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:44.987674 23232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:45.072805 23232 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:35771
I20260810 06:49:45.072901 23429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:35771 every 8 connection(s)
I20260810 06:49:45.075047 23232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:45.078734 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23232
I20260810 06:49:45.079087 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:45.079846 22574 external_mini_cluster.cc:1428] Setting key ad140c7e6273bda7bc46d61efed8d00f
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344584, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:45.111768 23433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:45.080815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:52039 (local address 127.22.11.190:33543)
0810 06:49:45.089528 (+ 8713us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:45.089533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:45.089550 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:49:45.089597 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:45.089601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:45.089703 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:45.089797 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:45.092724 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.093962 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.097255 (+ 3293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.098073 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.098200 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:45.104109 (+ 5909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:45.104115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:45.104118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:45.104148 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:45.108041 (+ 3893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.109009 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.109013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.109015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.109240 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.109892 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.109898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.109901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.110257 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:45.110441 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:45.111199 (+ 758us) server_negotiation.cc:299] Negotiation successful
0810 06:49:45.111395 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8587,"thread_start_us":8258,"threads_started":1}
I20260810 06:49:45.112777 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:45.079514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:33543 (local address 127.22.11.131:52039)
0810 06:49:45.081473 (+ 1959us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:45.081582 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:45.085397 (+ 3815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:45.090140 (+ 4743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:45.090153 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:45.091281 (+ 1128us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:45.092470 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:45.092501 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.094159 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.094165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:45.097095 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:45.097113 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.098448 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.098458 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:45.098913 (+ 455us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:45.099867 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:45.099902 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:45.103890 (+ 3988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:45.108213 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:45.108220 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:45.108232 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:45.108892 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:45.109372 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:45.109377 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:45.109382 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:45.109770 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:45.110417 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:45.110425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:45.110783 (+ 358us) client_negotiation.cc:769] Sending connection context
0810 06:49:45.111064 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 06:49:45.111457 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1576,"thread_start_us":1030,"threads_started":1}
I20260810 06:49:45.114209 23430 heartbeater.cc:347] Connected to a master server at 127.22.11.190:33543
I20260810 06:49:45.114544 23430 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:45.115594 23430 heartbeater.cc:511] Master 127.22.11.190:33543 requested a full tablet report, sending...
I20260810 06:49:45.117286 22663 ts_manager.cc:194] Registered new tserver with Master: 11d20185c9864d2e9c0321dc28dd95b2 (127.22.11.131:35771)
I20260810 06:49:45.118229 22663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:52039
I20260810 06:49:45.122820 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[22589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344585, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:49:45.185887 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:45.166402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54904 (local address 127.22.11.129:46079)
0810 06:49:45.167672 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:45.167679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:45.167707 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:49:45.167768 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:45.167774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:45.167980 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:45.168120 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:45.169806 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.171126 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.172837 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.173628 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.173728 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:45.179043 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:45.179056 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:45.179071 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:45.179130 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:45.182809 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.183531 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.183537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.183539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.183715 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.184147 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.184150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.184152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.184634 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:45.184950 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:45.185180 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 06:49:45.185406 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1118,"thread_start_us":882,"threads_started":1}
W20260810 06:49:45.187394 22844 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54904
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[22589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:49:45.238757 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:45.221738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.22.11.129:46079)
0810 06:49:45.222012 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:45.222018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:45.222087 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:49:45.222402 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:45.222407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:45.222476 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:45.222575 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:45.223954 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.224798 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.226612 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.227176 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.227295 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:45.232432 (+ 5137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:45.232438 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:45.232441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:45.232475 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:45.235997 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.236682 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.236688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.236692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.236881 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.237232 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.237236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.237238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.237479 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:45.237699 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:45.237997 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 06:49:45.238227 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:49:45.240751 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 22772
I20260810 06:49:45.256474 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:45.241733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.22.11.129:46079)
0810 06:49:45.241902 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:45.241906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:45.242513 (+ 607us) server_negotiation.cc:407] Connection header received
0810 06:49:45.242618 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:45.242622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:45.242676 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:45.242750 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:45.244240 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.245225 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.246762 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.247692 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.247825 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:45.250221 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:45.250228 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:45.250231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:45.250266 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:45.254139 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.254684 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.254688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.254690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.254888 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.255174 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.255178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.255180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.255461 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:45.255638 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:45.255825 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 06:49:45.256007 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:49:45.257172 22989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:45.266497 23223 heartbeater.cc:503] Master 127.22.11.190:33543 was elected leader, sending a full tablet report...
I20260810 06:49:45.346693 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 22772
I20260810 06:49:45.372680 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23005
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:49:45.388517 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:45.373631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.22.11.130:36607)
0810 06:49:45.374476 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:45.374483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:45.374508 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:49:45.374570 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:45.374576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:45.374785 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:45.374918 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:45.376045 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.377327 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.378534 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.379102 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.379208 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:45.383444 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:45.383456 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:45.383470 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:45.383503 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:45.386294 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.386883 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.386890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.386892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.387009 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.387324 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.387327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.387329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.387646 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:45.387899 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:45.388043 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 06:49:45.388237 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":521,"threads_started":1}
I20260810 06:49:45.389739 23215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:45.472453 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23005
I20260810 06:49:45.498783 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23232
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[22589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:49:45.513741 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:45.499760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.22.11.131:35771)
0810 06:49:45.500297 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:45.500303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:45.500333 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:49:45.500395 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:45.500402 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:45.500640 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:45.500793 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:45.501944 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.502884 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.504064 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:45.504671 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:45.504764 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:45.508570 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:45.508581 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:45.508595 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:45.508653 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:45.511418 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.512029 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.512044 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.512046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.512163 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:45.512435 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:45.512437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:45.512439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:45.512784 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:45.513003 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:45.513250 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 06:49:45.513436 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":238,"threads_started":1}
I20260810 06:49:45.515079 23423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:45.595580 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23232
I20260810 06:49:45.616932 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 22611
I20260810 06:49:45.618081 22719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:45.719940 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 22611
2026-08-10T06:49:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5570 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[23463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[23463](info): set up 2 sockets
Aug 10 06:49:45 dist-test-slave-f7th krb5kdc[23463](info): commencing operation
krb5kdc: starting...
W20260810 06:49:46.932582 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:49:47 dist-test-slave-f7th krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:49:47.109427 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45939
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:45939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:47.477064 23481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:47.477325 23481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:47.477454 23481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:47.484352 23481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:47.484450 23481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:47.484521 23481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:47.484598 23481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:47.484671 23481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:47.493969 23481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:45939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:47.495152 23481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:47.496395 23481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:47.507651 23493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:47.507624 23492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:47.507684 23495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:47.508404 23481 server_base.cc:1034] running on GCE node
I20260810 06:49:47.509155 23481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:47.510408 23481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:47.511591 23481 hybrid_clock.cc:648] HybridClock initialized: now 1786344587511536 us; error 44 us; skew 500 ppm
I20260810 06:49:47.511904 23481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:47 dist-test-slave-f7th krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344587, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:47.514825 23481 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:49:47.516160 23481 webserver.cc:460] Webserver started at http://127.22.11.190:40395/ using document root <none> and password file <none>
I20260810 06:49:47.516814 23481 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:47.516920 23481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:47.517176 23481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:47.519903 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "29fe031685434f3989b6fb2bdf8766ed"
format_stamp: "Formatted at 2026-08-10 06:49:47 on dist-test-slave-f7th"
server_key: "3e427a68be3069429d03be29f4f3a7f1"
server_key_iv: "d43c763d6cf2d3e08c68f56dd06f1936"
server_key_version: "encryptionkey@0"
I20260810 06:49:47.520581 23481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "29fe031685434f3989b6fb2bdf8766ed"
format_stamp: "Formatted at 2026-08-10 06:49:47 on dist-test-slave-f7th"
server_key: "3e427a68be3069429d03be29f4f3a7f1"
server_key_iv: "d43c763d6cf2d3e08c68f56dd06f1936"
server_key_version: "encryptionkey@0"
I20260810 06:49:47.525213 23481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:49:47.528308 23503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:47.529626 23481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:49:47.529784 23481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "29fe031685434f3989b6fb2bdf8766ed"
format_stamp: "Formatted at 2026-08-10 06:49:47 on dist-test-slave-f7th"
server_key: "3e427a68be3069429d03be29f4f3a7f1"
server_key_iv: "d43c763d6cf2d3e08c68f56dd06f1936"
server_key_version: "encryptionkey@0"
I20260810 06:49:47.529954 23481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:47.564482 23481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:47.568339 23481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:47.568608 23481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:47.585515 23481 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:45939
I20260810 06:49:47.585548 23588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:45939 every 8 connection(s)
I20260810 06:49:47.586907 23481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:47.590449 23589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:47.595252 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23481
I20260810 06:49:47.595486 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:49:47.596259 22574 external_mini_cluster.cc:1428] Setting key 14685042941a4368b7299403ded98ddb
I20260810 06:49:47.601439 23589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed: Bootstrap starting.
I20260810 06:49:47.604918 23589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:47.606135 23589 log.cc:824] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:47.609433 23589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed: No bootstrap required, opened a new log
Aug 10 06:49:47 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:47.618191 23589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29fe031685434f3989b6fb2bdf8766ed" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45939 } }
I20260810 06:49:47.618552 23589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:47.618655 23589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fe031685434f3989b6fb2bdf8766ed, State: Initialized, Role: FOLLOWER
I20260810 06:49:47.619205 23589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29fe031685434f3989b6fb2bdf8766ed" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45939 } }
I20260810 06:49:47.619341 23596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:47.598303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.22.11.190:45939)
0810 06:49:47.599117 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:47.599128 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:47.599173 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:49:47.600862 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:47.600886 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:47.601248 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:47.601563 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:47.603339 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.604606 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.606301 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:47.607264 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:47.607442 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:47.612065 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:47.612073 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:47.612089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:47.612141 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:47.615039 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:47.615776 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:47.615782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:47.615787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:47.615986 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:47.616458 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:47.616461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:47.616463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:47.617012 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:47.617218 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:47.617772 (+ 554us) server_negotiation.cc:299] Negotiation successful
0810 06:49:47.618110 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":348,"threads_started":1}
I20260810 06:49:47.619438 23589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:47.619558 23589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:47.619722 23589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:47.622006 23589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29fe031685434f3989b6fb2bdf8766ed" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45939 } }
I20260810 06:49:47.622522 23589 leader_election.cc:302] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29fe031685434f3989b6fb2bdf8766ed; no voters:
I20260810 06:49:47.623087 23589 leader_election.cc:288] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:47.623204 23599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:47.623835 23599 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [term 1 LEADER]: Becoming Leader. State: Replica: 29fe031685434f3989b6fb2bdf8766ed, State: Running, Role: LEADER
I20260810 06:49:47.624357 23599 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29fe031685434f3989b6fb2bdf8766ed" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45939 } }
I20260810 06:49:47.625162 23589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:47.627559 23601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29fe031685434f3989b6fb2bdf8766ed. Latest consensus state: current_term: 1 leader_uuid: "29fe031685434f3989b6fb2bdf8766ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29fe031685434f3989b6fb2bdf8766ed" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45939 } } }
I20260810 06:49:47.627526 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29fe031685434f3989b6fb2bdf8766ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29fe031685434f3989b6fb2bdf8766ed" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45939 } } }
I20260810 06:49:47.627986 23601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:47.627987 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:47.629622 23610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:47.634744 23610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:47.643500 23610 catalog_manager.cc:1347] Generated new cluster ID: 381ca65c0e254f0ab3c58814cacf3dcf
I20260810 06:49:47.643607 23610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:47.666514 23610 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:47.668383 23610 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:47.695885 23610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29fe031685434f3989b6fb2bdf8766ed: Generated new TSK 0
I20260810 06:49:47.696975 23610 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:49:47.745607 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45939
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:48.060750 23629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:48.061007 23629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:48.061122 23629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:48.068012 23629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:48.068113 23629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:48.068221 23629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:49:48.077474 23629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:48.078608 23629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:48.079873 23629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:48.092381 23642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:48.092450 23637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:48.092531 23639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:48.093405 23629 server_base.cc:1034] running on GCE node
I20260810 06:49:48.093976 23629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:48.094861 23629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:48.096066 23629 hybrid_clock.cc:648] HybridClock initialized: now 1786344588096009 us; error 51 us; skew 500 ppm
I20260810 06:49:48.096352 23629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:48 dist-test-slave-f7th krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344588, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:48.099293 23629 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:49:48.100606 23629 webserver.cc:460] Webserver started at http://127.22.11.129:38431/ using document root <none> and password file <none>
I20260810 06:49:48.101264 23629 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:48.101377 23629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:48.101671 23629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:48.104367 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "be3b9cf5505b4c7a9711c081ff99550e"
format_stamp: "Formatted at 2026-08-10 06:49:48 on dist-test-slave-f7th"
server_key: "309403035f2928b30d16d3a7f0efff1a"
server_key_iv: "6e93edf8d05150088511cd733dcb514c"
server_key_version: "encryptionkey@0"
I20260810 06:49:48.105149 23629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "be3b9cf5505b4c7a9711c081ff99550e"
format_stamp: "Formatted at 2026-08-10 06:49:48 on dist-test-slave-f7th"
server_key: "309403035f2928b30d16d3a7f0efff1a"
server_key_iv: "6e93edf8d05150088511cd733dcb514c"
server_key_version: "encryptionkey@0"
I20260810 06:49:48.110005 23629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 06:49:48.113281 23655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:48.114490 23629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:49:48.114647 23629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "be3b9cf5505b4c7a9711c081ff99550e"
format_stamp: "Formatted at 2026-08-10 06:49:48 on dist-test-slave-f7th"
server_key: "309403035f2928b30d16d3a7f0efff1a"
server_key_iv: "6e93edf8d05150088511cd733dcb514c"
server_key_version: "encryptionkey@0"
I20260810 06:49:48.114820 23629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:48.133780 23629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:48.137518 23629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:48.137748 23629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:48.138588 23629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:48.139822 23629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:48.139894 23629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:48.139986 23629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:48.140027 23629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:48.169018 23629 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:45275
I20260810 06:49:48.169070 23831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:45275 every 8 connection(s)
I20260810 06:49:48.170275 23629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:48.171142 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23629
I20260810 06:49:48.171371 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:48.172192 22574 external_mini_cluster.cc:1428] Setting key 1abe292975030299273cf98ddac5d530
Aug 10 06:49:48 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344588, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
I20260810 06:49:48.190467 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:48.173226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:44879 (local address 127.22.11.190:45939)
0810 06:49:48.173682 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:48.173687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:48.174790 (+ 1103us) server_negotiation.cc:407] Connection header received
0810 06:49:48.176633 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:48.176638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:48.176715 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:48.176827 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:48.178241 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.179203 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.180676 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.181280 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.181407 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:48.185060 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:48.185065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:48.185068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:48.185100 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:48.187583 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:48.188245 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:48.188250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:48.188254 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:48.188380 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:48.188824 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:48.188830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:48.188833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:48.189109 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:48.189240 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:48.189879 (+ 639us) server_negotiation.cc:299] Negotiation successful
0810 06:49:48.190095 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
I20260810 06:49:48.191330 23838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:48.173774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45939 (local address 127.22.11.129:44879)
0810 06:49:48.174623 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:48.174673 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:48.176358 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:48.177007 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:48.177015 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:48.177519 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:48.178077 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:48.178095 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.179324 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.179327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:48.180510 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:48.180522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.181469 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.181476 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:48.181746 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:48.182380 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:48.182413 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:48.184824 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:48.187707 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:48.187716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:48.187731 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:48.188135 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:48.188483 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:48.188488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:48.188492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:48.188721 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:48.189231 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:48.189239 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:48.189549 (+ 310us) client_negotiation.cc:769] Sending connection context
0810 06:49:48.189785 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 06:49:48.190266 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":348,"threads_started":1}
I20260810 06:49:48.193006 23834 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
I20260810 06:49:48.193287 23834 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:48.194129 23834 heartbeater.cc:511] Master 127.22.11.190:45939 requested a full tablet report, sending...
I20260810 06:49:48.196442 23529 ts_manager.cc:194] Registered new tserver with Master: be3b9cf5505b4c7a9711c081ff99550e (127.22.11.129:45275)
I20260810 06:49:48.197885 23529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:44879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:49:48.213789 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45939
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:48.530548 23845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:48.530812 23845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:48.530952 23845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:48.538157 23845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:48.538262 23845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:48.538412 23845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:49:48.547993 23845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:48.549304 23845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:48.550619 23845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:48.563208 23857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:48.563239 23858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:48.563167 23865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:48.564014 23845 server_base.cc:1034] running on GCE node
I20260810 06:49:48.564581 23845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:48.565471 23845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:48.566707 23845 hybrid_clock.cc:648] HybridClock initialized: now 1786344588566650 us; error 39 us; skew 500 ppm
I20260810 06:49:48.567056 23845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:48 dist-test-slave-f7th krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344588, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:48.570147 23845 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:49:48.571491 23845 webserver.cc:460] Webserver started at http://127.22.11.130:46415/ using document root <none> and password file <none>
I20260810 06:49:48.572116 23845 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:48.572223 23845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:48.572588 23845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:48.575394 23845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "f2509bb5b93742f890a40a7680644583"
format_stamp: "Formatted at 2026-08-10 06:49:48 on dist-test-slave-f7th"
server_key: "6f72d634c23f9b00cb30a1c1cf6ef66a"
server_key_iv: "3dc4f103fc7edc351e8bb95db45ee114"
server_key_version: "encryptionkey@0"
I20260810 06:49:48.576133 23845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "f2509bb5b93742f890a40a7680644583"
format_stamp: "Formatted at 2026-08-10 06:49:48 on dist-test-slave-f7th"
server_key: "6f72d634c23f9b00cb30a1c1cf6ef66a"
server_key_iv: "3dc4f103fc7edc351e8bb95db45ee114"
server_key_version: "encryptionkey@0"
I20260810 06:49:48.581101 23845 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 06:49:48.584353 23875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:48.585685 23845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:48.585830 23845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "f2509bb5b93742f890a40a7680644583"
format_stamp: "Formatted at 2026-08-10 06:49:48 on dist-test-slave-f7th"
server_key: "6f72d634c23f9b00cb30a1c1cf6ef66a"
server_key_iv: "3dc4f103fc7edc351e8bb95db45ee114"
server_key_version: "encryptionkey@0"
I20260810 06:49:48.586005 23845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:48.603469 23845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:48.607250 23845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:48.607504 23845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:48.608398 23845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:48.609752 23845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:48.609823 23845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:48.609964 23845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:48.610030 23845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:48.640116 23845 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:36553
I20260810 06:49:48.640183 24037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:36553 every 8 connection(s)
I20260810 06:49:48.641400 23845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:48.649468 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23845
I20260810 06:49:48.649646 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:48.650063 22574 external_mini_cluster.cc:1428] Setting key 4558fc1ee815b12ae11a8bebe544dc40
Aug 10 06:49:48 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344588, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
I20260810 06:49:48.668375 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:48.643969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:45859 (local address 127.22.11.190:45939)
0810 06:49:48.644173 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:48.644178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:48.645543 (+ 1365us) server_negotiation.cc:407] Connection header received
0810 06:49:48.647449 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:48.647456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:48.647533 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:48.647622 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:48.649192 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.650022 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.651721 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.652244 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.652348 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:48.662735 (+ 10387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:48.662741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:48.662744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:48.662777 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:48.665336 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:48.666035 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:48.666040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:48.666043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:48.666178 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:48.666598 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:48.666601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:48.666603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:48.666936 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:48.667084 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:48.667797 (+ 713us) server_negotiation.cc:299] Negotiation successful
0810 06:49:48.668023 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:49:48.669363 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:48.644434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45939 (local address 127.22.11.130:45859)
0810 06:49:48.645306 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:48.645359 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:48.647156 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:48.647860 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:48.647869 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:48.648391 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:48.649014 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:48.649032 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.650178 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.650183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:48.651569 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:48.651582 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:48.652507 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:48.652516 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:48.652834 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:48.660100 (+ 7266us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:48.660131 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:48.662486 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:48.665477 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:48.665486 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:48.665504 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:48.665908 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:48.666285 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:48.666290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:48.666294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:48.666476 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:48.667070 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:48.667078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:48.667436 (+ 358us) client_negotiation.cc:769] Sending connection context
0810 06:49:48.667678 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 06:49:48.668214 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":362,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
I20260810 06:49:48.670763 24038 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45939
I20260810 06:49:48.671096 24038 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:48.671967 24038 heartbeater.cc:511] Master 127.22.11.190:45939 requested a full tablet report, sending...
I20260810 06:49:48.673508 23529 ts_manager.cc:194] Registered new tserver with Master: f2509bb5b93742f890a40a7680644583 (127.22.11.130:36553)
I20260810 06:49:48.674312 23529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:45859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:49:48.690045 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45939
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:49.015385 24047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:49.015666 24047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:49.015769 24047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:49.022939 24047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:49.023041 24047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:49.023166 24047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:49:49.032564 24047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45939
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:49.033814 24047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:49.035130 24047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:49.047384 24056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:49.047436 24058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:49.047472 24053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:49.048429 24047 server_base.cc:1034] running on GCE node
I20260810 06:49:49.049064 24047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:49.049921 24047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:49.051110 24047 hybrid_clock.cc:648] HybridClock initialized: now 1786344589051063 us; error 36 us; skew 500 ppm
I20260810 06:49:49.051404 24047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:49.054466 24047 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:49:49.055799 24047 webserver.cc:460] Webserver started at http://127.22.11.131:39057/ using document root <none> and password file <none>
I20260810 06:49:49.056418 24047 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:49.056530 24047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:49.056856 24047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:49.059631 24047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "c6b1694ec73d4fcd9468afa3fab50145"
format_stamp: "Formatted at 2026-08-10 06:49:49 on dist-test-slave-f7th"
server_key: "5b4e46c1dd98214358ef41ae57532fb7"
server_key_iv: "20909efd0fdfbc6ad19c82bf7c105005"
server_key_version: "encryptionkey@0"
I20260810 06:49:49.060364 24047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "c6b1694ec73d4fcd9468afa3fab50145"
format_stamp: "Formatted at 2026-08-10 06:49:49 on dist-test-slave-f7th"
server_key: "5b4e46c1dd98214358ef41ae57532fb7"
server_key_iv: "20909efd0fdfbc6ad19c82bf7c105005"
server_key_version: "encryptionkey@0"
I20260810 06:49:49.065387 24047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:49:49.068931 24071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:49.070235 24047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:49:49.070384 24047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "c6b1694ec73d4fcd9468afa3fab50145"
format_stamp: "Formatted at 2026-08-10 06:49:49 on dist-test-slave-f7th"
server_key: "5b4e46c1dd98214358ef41ae57532fb7"
server_key_iv: "20909efd0fdfbc6ad19c82bf7c105005"
server_key_version: "encryptionkey@0"
I20260810 06:49:49.070582 24047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:49.087671 24047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:49.095561 24047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:49.095817 24047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:49.096725 24047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:49:49.098016 24047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:49.098085 24047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:49.098203 24047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:49.098253 24047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:49.126919 24047 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:41991
I20260810 06:49:49.126959 24255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:41991 every 8 connection(s)
I20260810 06:49:49.128129 24047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:49.135455 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24047
I20260810 06:49:49.135644 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:49.136047 22574 external_mini_cluster.cc:1428] Setting key 71646cebf7b20b6972c56b847d79059d
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:49.146289 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.130569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:48167 (local address 127.22.11.190:45939)
0810 06:49:49.130780 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.130786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.131936 (+ 1150us) server_negotiation.cc:407] Connection header received
0810 06:49:49.133686 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.133690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.133775 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.133874 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.135282 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.136064 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.137585 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.137986 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.138102 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.141353 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.141358 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.141362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.141394 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.143863 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.144467 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.144470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.144473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.144582 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.144956 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.144958 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.144960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.145236 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.145351 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.145888 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.146053 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:49:49.147183 24259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.131098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45939 (local address 127.22.11.131:48167)
0810 06:49:49.131790 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:49.131826 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:49.133442 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:49.134081 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:49.134089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:49.134580 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:49.135126 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.135150 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.136215 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.136220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.137444 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.137457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.138203 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.138210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.138439 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.139032 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:49.139055 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:49.141159 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:49.143967 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:49.143973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:49.143983 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:49.144379 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:49.144696 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:49.144699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:49.144701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:49.144873 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:49.145325 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:49.145330 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:49.145618 (+ 288us) client_negotiation.cc:769] Sending connection context
0810 06:49:49.145824 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 06:49:49.146185 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":254,"threads_started":1}
I20260810 06:49:49.148380 24256 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45939
I20260810 06:49:49.148684 24256 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:49.149371 24256 heartbeater.cc:511] Master 127.22.11.190:45939 requested a full tablet report, sending...
I20260810 06:49:49.150771 23527 ts_manager.cc:194] Registered new tserver with Master: c6b1694ec73d4fcd9468afa3fab50145 (127.22.11.131:41991)
I20260810 06:49:49.151602 23527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:48167
I20260810 06:49:49.162462 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:49.181932 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.169427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.22.11.190:45939)
0810 06:49:49.169600 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.169607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.170158 (+ 551us) server_negotiation.cc:407] Connection header received
0810 06:49:49.170336 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.170339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.170397 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.170486 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.172379 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.173118 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.174773 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.175243 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.175329 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.177525 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.177530 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.177533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.177559 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.179774 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.180325 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.180328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.180333 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.180438 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.180749 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.180752 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.180754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.181056 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.181244 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.181522 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.181667 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:49:49.189874 23527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:49.197867 23527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:49.200702 23834 heartbeater.cc:503] Master 127.22.11.190:45939 was elected leader, sending a full tablet report...
I20260810 06:49:49.226985 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.212309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:36553 (local address 127.0.0.1:59906)
0810 06:49:49.216379 (+ 4070us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:49.216397 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:49.216603 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:49.217572 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:49.217580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:49.217611 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:49.218601 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.218623 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.220823 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.220829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.223322 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.223340 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.225764 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.225771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.226032 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.226086 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:49:49.226248 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 06:49:49.226463 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3933,"threads_started":1}
I20260810 06:49:49.228147 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.212758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59906 (local address 127.22.11.130:36553)
0810 06:49:49.215907 (+ 3149us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.215917 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.216486 (+ 569us) server_negotiation.cc:407] Connection header received
0810 06:49:49.216828 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.216838 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.217086 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.217259 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:49.218810 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.220628 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.223534 (+ 2906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.225218 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.225333 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.225554 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.227558 (+ 2004us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.227775 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2975,"thread_start_us":2815,"threads_started":1}
I20260810 06:49:49.233459 24278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.212239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:45275 (local address 127.0.0.1:41784)
0810 06:49:49.212963 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:49.213003 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:49.213225 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:49.226081 (+ 12856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:49.226087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:49.226135 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:49.226634 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.226645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.229327 (+ 2682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.229332 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.231113 (+ 1781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.231126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.232537 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.232544 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.232753 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.232788 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:49:49.232909 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 06:49:49.233038 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":43,"thread_start_us":759,"threads_started":1}
I20260810 06:49:49.234359 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.212832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41784 (local address 127.22.11.129:45275)
0810 06:49:49.225458 (+ 12626us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.225467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.225498 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:49:49.225598 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.225606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.225843 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.226982 (+ 1139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:49.227239 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.228862 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.231263 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.232297 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.232444 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.232721 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.233883 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.234051 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12190,"thread_start_us":12222,"threads_started":1}
I20260810 06:49:49.234059 23943 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce18e5b92334056845b1d6a3c606307 (DEFAULT_TABLE table=test-table [id=b0fd7968745f42cf9877f26ceb198e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:49.235738 23943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce18e5b92334056845b1d6a3c606307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:49.237478 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.215258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:41991 (local address 127.0.0.1:43404)
0810 06:49:49.227268 (+ 12010us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:49.227288 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:49.227384 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:49.229766 (+ 2382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:49.229771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:49.229801 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:49.230123 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.230135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.232010 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.232016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.233960 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.233975 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.236749 (+ 2774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.236755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.236934 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.236967 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:49:49.237064 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:49:49.237186 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11905,"mutex_wait_us":23,"thread_start_us":841,"threads_started":1}
I20260810 06:49:49.237749 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.216041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.22.11.131:41991)
0810 06:49:49.229208 (+ 13167us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.229216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.229249 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:49:49.229344 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.229351 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.229594 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.229778 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:49.230273 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.231848 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.234109 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.235174 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.235294 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.235524 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.237131 (+ 1607us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.237289 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12945,"thread_start_us":12721,"threads_started":1}
I20260810 06:49:49.237507 23733 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce18e5b92334056845b1d6a3c606307 (DEFAULT_TABLE table=test-table [id=b0fd7968745f42cf9877f26ceb198e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:49.238785 23733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce18e5b92334056845b1d6a3c606307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:49.241395 24153 tablet_service.cc:1467] Processing CreateTablet for tablet 8ce18e5b92334056845b1d6a3c606307 (DEFAULT_TABLE table=test-table [id=b0fd7968745f42cf9877f26ceb198e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:49.242785 24153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ce18e5b92334056845b1d6a3c606307. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:49.252338 24287 tablet_bootstrap.cc:492] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583: Bootstrap starting.
I20260810 06:49:49.255394 24287 tablet_bootstrap.cc:654] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:49.256664 24287 log.cc:824] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:49.256785 24290 tablet_bootstrap.cc:492] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145: Bootstrap starting.
I20260810 06:49:49.259333 24292 tablet_bootstrap.cc:492] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e: Bootstrap starting.
I20260810 06:49:49.260210 24287 tablet_bootstrap.cc:492] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583: No bootstrap required, opened a new log
I20260810 06:49:49.260394 24290 tablet_bootstrap.cc:654] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:49.260460 24287 ts_tablet_manager.cc:1397] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260810 06:49:49.261725 24290 log.cc:824] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:49.262915 24292 tablet_bootstrap.cc:654] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:49.264153 24292 log.cc:824] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:49.268172 24287 raft_consensus.cc:348] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.268509 24287 raft_consensus.cc:374] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:49.268668 24287 raft_consensus.cc:729] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2509bb5b93742f890a40a7680644583, State: Initialized, Role: FOLLOWER
I20260810 06:49:49.269367 24287 consensus_queue.cc:260] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.270509 24038 heartbeater.cc:503] Master 127.22.11.190:45939 was elected leader, sending a full tablet report...
I20260810 06:49:49.270977 24290 tablet_bootstrap.cc:492] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145: No bootstrap required, opened a new log
I20260810 06:49:49.271063 24292 tablet_bootstrap.cc:492] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e: No bootstrap required, opened a new log
I20260810 06:49:49.271261 24290 ts_tablet_manager.cc:1397] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260810 06:49:49.271333 24292 ts_tablet_manager.cc:1397] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260810 06:49:49.276005 24287 ts_tablet_manager.cc:1428] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583: Time spent starting tablet: real 0.015s user 0.007s sys 0.003s
I20260810 06:49:49.282150 24292 raft_consensus.cc:348] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.282106 24290 raft_consensus.cc:348] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.282771 24292 raft_consensus.cc:374] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:49.282730 24290 raft_consensus.cc:374] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:49.282979 24290 raft_consensus.cc:729] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6b1694ec73d4fcd9468afa3fab50145, State: Initialized, Role: FOLLOWER
I20260810 06:49:49.282965 24292 raft_consensus.cc:729] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be3b9cf5505b4c7a9711c081ff99550e, State: Initialized, Role: FOLLOWER
I20260810 06:49:49.283813 24290 consensus_queue.cc:260] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.284044 24292 consensus_queue.cc:260] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.285468 24256 heartbeater.cc:503] Master 127.22.11.190:45939 was elected leader, sending a full tablet report...
I20260810 06:49:49.286402 24290 ts_tablet_manager.cc:1428] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260810 06:49:49.286434 24292 ts_tablet_manager.cc:1428] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e: Time spent starting tablet: real 0.015s user 0.009s sys 0.006s
I20260810 06:49:49.322059 24296 raft_consensus.cc:482] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:49.322501 24296 raft_consensus.cc:504] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.325539 24296 leader_election.cc:288] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be3b9cf5505b4c7a9711c081ff99550e (127.22.11.129:45275), c6b1694ec73d4fcd9468afa3fab50145 (127.22.11.131:41991)
I20260810 06:49:49.338441 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.326631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:55061 (local address 127.22.11.131:41991)
0810 06:49:49.326914 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.326919 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.328978 (+ 2059us) server_negotiation.cc:407] Connection header received
0810 06:49:49.329098 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.329103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.329179 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.329264 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:49.330229 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.331798 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.335730 (+ 3932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.337050 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.337237 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.337323 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.337927 (+ 604us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.338065 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:49:49.338526 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.326145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:58387 (local address 127.22.11.129:45275)
0810 06:49:49.326455 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.326461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.328783 (+ 2322us) server_negotiation.cc:407] Connection header received
0810 06:49:49.329100 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.329106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.329179 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.329239 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:49.331082 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.332100 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.335680 (+ 3580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.337129 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.337326 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.337431 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.338072 (+ 641us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.338207 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:49:49.338941 24302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.326320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:41991 (local address 127.22.11.130:55061)
0810 06:49:49.328874 (+ 2554us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:49.328890 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:49.328998 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:49.329331 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:49.329334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:49.329357 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:49.330062 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.330073 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.331953 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.331959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.335541 (+ 3582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.335555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.337513 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.337521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.337763 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.337793 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:49:49.337867 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:49:49.337959 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2425,"mutex_wait_us":24,"thread_start_us":1889,"threads_started":1}
I20260810 06:49:49.339058 24301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.325776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:45275 (local address 127.22.11.130:58387)
0810 06:49:49.328665 (+ 2889us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:49.328686 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:49.328894 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:49.329501 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:49.329505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:49.329558 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:49.330940 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.330958 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.332271 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.332274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.335491 (+ 3217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:49.335505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.337608 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.337615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:49.337895 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.337928 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:49:49.338017 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:49:49.338132 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2449,"mutex_wait_us":2,"thread_start_us":1772,"threads_started":1}
I20260810 06:49:49.340499 24188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18e5b92334056845b1d6a3c606307" candidate_uuid: "f2509bb5b93742f890a40a7680644583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b1694ec73d4fcd9468afa3fab50145" is_pre_election: true
I20260810 06:49:49.340497 23765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18e5b92334056845b1d6a3c606307" candidate_uuid: "f2509bb5b93742f890a40a7680644583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be3b9cf5505b4c7a9711c081ff99550e" is_pre_election: true
I20260810 06:49:49.341377 23765 raft_consensus.cc:2454] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2509bb5b93742f890a40a7680644583 in term 0.
I20260810 06:49:49.341501 24188 raft_consensus.cc:2454] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2509bb5b93742f890a40a7680644583 in term 0.
I20260810 06:49:49.342500 23876 leader_election.cc:302] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be3b9cf5505b4c7a9711c081ff99550e, f2509bb5b93742f890a40a7680644583; no voters:
I20260810 06:49:49.343026 24296 raft_consensus.cc:2781] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:49.343326 24296 raft_consensus.cc:482] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:49.343511 24296 raft_consensus.cc:3037] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:49.346814 24296 raft_consensus.cc:504] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.347697 24296 leader_election.cc:288] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [CANDIDATE]: Term 1 election: Requested vote from peers be3b9cf5505b4c7a9711c081ff99550e (127.22.11.129:45275), c6b1694ec73d4fcd9468afa3fab50145 (127.22.11.131:41991)
I20260810 06:49:49.348141 23765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18e5b92334056845b1d6a3c606307" candidate_uuid: "f2509bb5b93742f890a40a7680644583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be3b9cf5505b4c7a9711c081ff99550e"
I20260810 06:49:49.348227 24188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ce18e5b92334056845b1d6a3c606307" candidate_uuid: "f2509bb5b93742f890a40a7680644583" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6b1694ec73d4fcd9468afa3fab50145"
I20260810 06:49:49.348408 23765 raft_consensus.cc:3037] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:49.348445 24188 raft_consensus.cc:3037] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:49.351761 23765 raft_consensus.cc:2454] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2509bb5b93742f890a40a7680644583 in term 1.
I20260810 06:49:49.352247 23876 leader_election.cc:302] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: be3b9cf5505b4c7a9711c081ff99550e, f2509bb5b93742f890a40a7680644583; no voters:
I20260810 06:49:49.352311 24188 raft_consensus.cc:2454] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2509bb5b93742f890a40a7680644583 in term 1.
I20260810 06:49:49.352665 24296 raft_consensus.cc:2781] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:49.353618 24296 raft_consensus.cc:686] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [term 1 LEADER]: Becoming Leader. State: Replica: f2509bb5b93742f890a40a7680644583, State: Running, Role: LEADER
I20260810 06:49:49.354395 24296 consensus_queue.cc:237] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } }
I20260810 06:49:49.360406 23529 catalog_manager.cc:5495] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2509bb5b93742f890a40a7680644583 (127.22.11.130). New cstate: current_term: 1 leader_uuid: "f2509bb5b93742f890a40a7680644583" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be3b9cf5505b4c7a9711c081ff99550e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 45275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2509bb5b93742f890a40a7680644583" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6b1694ec73d4fcd9468afa3fab50145" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41991 } health_report { overall_health: UNKNOWN } } }
W20260810 06:49:49.379801 24257 tablet.cc:2369] T 8ce18e5b92334056845b1d6a3c606307 P c6b1694ec73d4fcd9468afa3fab50145: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344587, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:49:49.391445 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.367772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.22.11.129:45275)
0810 06:49:49.367973 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.367980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.369526 (+ 1546us) server_negotiation.cc:407] Connection header received
0810 06:49:49.369687 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.369690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.369753 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.369868 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.373086 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.374276 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.376290 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.377149 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.377243 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.383724 (+ 6481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.383739 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.383754 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.383786 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.387903 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.388728 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.388735 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.388738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.388954 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.389377 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.389380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.389382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.389939 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.390364 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.390708 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.390932 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 06:49:49.393361 24041 tablet.cc:2369] T 8ce18e5b92334056845b1d6a3c606307 P f2509bb5b93742f890a40a7680644583: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260810 06:49:49.422648 23836 tablet.cc:2369] T 8ce18e5b92334056845b1d6a3c606307 P be3b9cf5505b4c7a9711c081ff99550e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:49:49.446736 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.428799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.22.11.129:45275)
0810 06:49:49.429022 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.429028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.429325 (+ 297us) server_negotiation.cc:407] Connection header received
0810 06:49:49.429519 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.429523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.429600 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.429683 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.432143 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.432914 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.434833 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.435214 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.435300 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.440774 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.440778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.440782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.440823 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.444050 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.444714 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.444718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.444720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.444938 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.445445 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.445447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.445449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.445796 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.446025 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.446217 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.446393 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:49:49.499848 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.481036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.22.11.129:45275)
0810 06:49:49.481210 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.481217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.481580 (+ 363us) server_negotiation.cc:407] Connection header received
0810 06:49:49.481756 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.481760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.481827 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.481909 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.483207 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.484216 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.485890 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.486701 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.486819 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.492660 (+ 5841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.492669 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.492674 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.492738 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.496327 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.497187 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.497193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.497196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.497357 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.497791 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.497794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.497796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.498310 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.498715 (+ 405us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.498974 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.499268 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 06:49:49.502147 23704 tablet_service.cc:2908] Rejecting scan request for tablet 8ce18e5b92334056845b1d6a3c606307: Uninitialized: clean time has not yet been initialized
I20260810 06:49:49.505720 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23629
I20260810 06:49:49.520134 24283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.506508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41828 (local address 127.22.11.129:45275)
0810 06:49:49.506667 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.506673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.506977 (+ 304us) server_negotiation.cc:407] Connection header received
0810 06:49:49.507099 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.507102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.507168 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.507241 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.508234 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.508905 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.510209 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.510682 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.510829 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.514036 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.514042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.514045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.514074 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.517419 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.518156 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.518160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.518162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.518352 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.518736 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.518739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.518741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.519104 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.519430 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.519604 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.519766 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:49:49.521019 23826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:49.610286 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23629
I20260810 06:49:49.637144 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23845
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:49:49.653358 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.638008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.22.11.130:36553)
0810 06:49:49.638250 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.638254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.638272 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:49:49.638328 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.638333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.638433 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.638518 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.639742 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.640721 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.641994 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.642893 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.643007 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.647789 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.647803 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.647821 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.647858 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.650783 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.651536 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.651543 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.651545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.651674 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.652001 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.652004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.652006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.652342 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.652566 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.652893 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.653064 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:49:49.653968 24032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:49.742601 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23845
I20260810 06:49:49.768389 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24047
Aug 10 06:49:49 dist-test-slave-f7th krb5kdc[23463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:49:49.781988 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:49.769172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.22.11.131:41991)
0810 06:49:49.769389 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:49.769393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:49.769422 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:49:49.769564 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:49.769568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:49.769633 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:49.769717 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:49.770881 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.771606 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.772936 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:49.773304 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:49.773395 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:49.777311 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:49.777324 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:49.777339 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:49.777370 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:49.779785 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.780344 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.780351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.780353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.780475 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:49.780796 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:49.780799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:49.780801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:49.781161 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:49.781385 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:49.781565 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 06:49:49.781750 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 06:49:49.782678 24248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:49.870311 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24047
I20260810 06:49:49.888675 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23481
I20260810 06:49:49.889817 23580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:49.995359 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23481
2026-08-10T06:49:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4275 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:50 dist-test-slave-f7th krb5kdc[24331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:50 dist-test-slave-f7th krb5kdc[24331](info): set up 2 sockets
Aug 10 06:49:50 dist-test-slave-f7th krb5kdc[24331](info): commencing operation
krb5kdc: starting...
W20260810 06:49:51.215457 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.004s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:49:51 dist-test-slave-f7th krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:49:51.389689 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40713
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:51.702561 24360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:51.702826 24360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:51.702935 24360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:51.709882 24360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:51.709980 24360 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:49:51.710027 24360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:51.710090 24360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:51.710139 24360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:51.710198 24360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:51.720609 24360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40713
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:51.721820 24360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:51.723133 24360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:51.734225 24367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:51.734189 24370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:51.734298 24368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:51.734941 24360 server_base.cc:1034] running on GCE node
I20260810 06:49:51.735677 24360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:51.736969 24360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:51.738181 24360 hybrid_clock.cc:648] HybridClock initialized: now 1786344591738127 us; error 26 us; skew 500 ppm
I20260810 06:49:51.738492 24360 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:51 dist-test-slave-f7th krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344591, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:51.741446 24360 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:49:51.742794 24360 webserver.cc:460] Webserver started at http://127.22.11.190:42449/ using document root <none> and password file <none>
I20260810 06:49:51.743423 24360 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:51.743510 24360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:51.743779 24360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:51.746596 24360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "c11942ab9e5b4e89b605876a13eaaecc"
format_stamp: "Formatted at 2026-08-10 06:49:51 on dist-test-slave-f7th"
server_key: "f3bcc18f0fc4bbc7f691eedc25078d91"
server_key_iv: "e765d6fd475b14cbc6f6d1743ca0ae69"
server_key_version: "encryptionkey@0"
I20260810 06:49:51.747320 24360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "c11942ab9e5b4e89b605876a13eaaecc"
format_stamp: "Formatted at 2026-08-10 06:49:51 on dist-test-slave-f7th"
server_key: "f3bcc18f0fc4bbc7f691eedc25078d91"
server_key_iv: "e765d6fd475b14cbc6f6d1743ca0ae69"
server_key_version: "encryptionkey@0"
I20260810 06:49:51.752153 24360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:49:51.755426 24382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:51.756657 24360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:49:51.756808 24360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "c11942ab9e5b4e89b605876a13eaaecc"
format_stamp: "Formatted at 2026-08-10 06:49:51 on dist-test-slave-f7th"
server_key: "f3bcc18f0fc4bbc7f691eedc25078d91"
server_key_iv: "e765d6fd475b14cbc6f6d1743ca0ae69"
server_key_version: "encryptionkey@0"
I20260810 06:49:51.756992 24360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:51.772504 24360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:51.776129 24360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:51.776374 24360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:51.796383 24360 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:40713
I20260810 06:49:51.796404 24472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:40713 every 8 connection(s)
I20260810 06:49:51.797715 24360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:51.798990 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24360
I20260810 06:49:51.799235 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:49:51.800091 22574 external_mini_cluster.cc:1428] Setting key d996eba525ee91eddcbbc4f60f2da7bb
I20260810 06:49:51.802438 24473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:49:51 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:51.815073 24473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc: Bootstrap starting.
I20260810 06:49:51.818181 24473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:51.819181 24473 log.cc:824] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:51.821977 24473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc: No bootstrap required, opened a new log
I20260810 06:49:51.822310 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:51.802103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.22.11.190:40713)
0810 06:49:51.802901 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:51.802912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:51.802955 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:49:51.804594 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:51.804640 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:51.805044 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:51.805366 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:51.807100 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:51.808457 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:51.810125 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:51.810774 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:51.810932 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:51.815375 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:51.815382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:51.815396 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:51.815432 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:51.818405 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:51.819036 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:51.819045 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:51.819053 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:51.819215 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:51.819576 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:51.819582 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:51.819585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:51.820194 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:51.820506 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:51.820946 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 06:49:51.821340 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":349,"threads_started":1}
I20260810 06:49:51.829748 24473 raft_consensus.cc:348] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c11942ab9e5b4e89b605876a13eaaecc" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40713 } }
I20260810 06:49:51.830094 24473 raft_consensus.cc:374] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:51.830190 24473 raft_consensus.cc:729] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c11942ab9e5b4e89b605876a13eaaecc, State: Initialized, Role: FOLLOWER
I20260810 06:49:51.830752 24473 consensus_queue.cc:260] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c11942ab9e5b4e89b605876a13eaaecc" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40713 } }
I20260810 06:49:51.830974 24473 raft_consensus.cc:388] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:51.831120 24473 raft_consensus.cc:482] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:51.831279 24473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:51.833511 24473 raft_consensus.cc:504] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c11942ab9e5b4e89b605876a13eaaecc" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40713 } }
I20260810 06:49:51.834003 24473 leader_election.cc:302] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c11942ab9e5b4e89b605876a13eaaecc; no voters:
I20260810 06:49:51.834525 24473 leader_election.cc:288] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:51.834657 24485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:51.835285 24485 raft_consensus.cc:686] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [term 1 LEADER]: Becoming Leader. State: Replica: c11942ab9e5b4e89b605876a13eaaecc, State: Running, Role: LEADER
I20260810 06:49:51.835773 24485 consensus_queue.cc:237] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c11942ab9e5b4e89b605876a13eaaecc" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40713 } }
I20260810 06:49:51.836463 24473 sys_catalog.cc:564] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:51.838470 24489 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c11942ab9e5b4e89b605876a13eaaecc. Latest consensus state: current_term: 1 leader_uuid: "c11942ab9e5b4e89b605876a13eaaecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c11942ab9e5b4e89b605876a13eaaecc" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40713 } } }
I20260810 06:49:51.838488 24488 sys_catalog.cc:455] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c11942ab9e5b4e89b605876a13eaaecc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c11942ab9e5b4e89b605876a13eaaecc" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40713 } } }
I20260810 06:49:51.839354 24488 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:51.839350 24489 sys_catalog.cc:458] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:51.840518 24494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:51.846030 24494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:51.855074 24494 catalog_manager.cc:1347] Generated new cluster ID: e7ba640cb3584473b46250e7a1ad999e
I20260810 06:49:51.855237 24494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:51.879491 24494 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:51.880865 24494 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:51.887113 24494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc: Generated new TSK 0
I20260810 06:49:51.887869 24494 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:49:51.950359 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40713
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:52.296584 24516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:52.296913 24516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:52.297073 24516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:52.304292 24516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:52.304404 24516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:52.304538 24516 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:49:52.304673 24516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:49:52.314702 24516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:40713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:52.315955 24516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:52.317301 24516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:52.330480 24533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:52.330458 24537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:52.330720 24532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:52.333884 24516 server_base.cc:1034] running on GCE node
I20260810 06:49:52.334468 24516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:52.335330 24516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:52.336568 24516 hybrid_clock.cc:648] HybridClock initialized: now 1786344592336502 us; error 39 us; skew 500 ppm
I20260810 06:49:52.336884 24516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:52 dist-test-slave-f7th krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344592, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:52.340167 24516 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:49:52.341667 24516 webserver.cc:460] Webserver started at http://127.22.11.129:36115/ using document root <none> and password file <none>
I20260810 06:49:52.342351 24516 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:52.342450 24516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:52.342756 24516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:52.345600 24516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "a7172c12d80d4df48ec0eee075fa0ab0"
format_stamp: "Formatted at 2026-08-10 06:49:52 on dist-test-slave-f7th"
server_key: "8e943c29b68247938b0eb7cf6d7c7d0f"
server_key_iv: "53f231a1d989f39b61eb0a0e7f2135fc"
server_key_version: "encryptionkey@0"
I20260810 06:49:52.346356 24516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "a7172c12d80d4df48ec0eee075fa0ab0"
format_stamp: "Formatted at 2026-08-10 06:49:52 on dist-test-slave-f7th"
server_key: "8e943c29b68247938b0eb7cf6d7c7d0f"
server_key_iv: "53f231a1d989f39b61eb0a0e7f2135fc"
server_key_version: "encryptionkey@0"
I20260810 06:49:52.351223 24516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:49:52.354416 24545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:52.355593 24516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:49:52.355741 24516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "a7172c12d80d4df48ec0eee075fa0ab0"
format_stamp: "Formatted at 2026-08-10 06:49:52 on dist-test-slave-f7th"
server_key: "8e943c29b68247938b0eb7cf6d7c7d0f"
server_key_iv: "53f231a1d989f39b61eb0a0e7f2135fc"
server_key_version: "encryptionkey@0"
I20260810 06:49:52.355938 24516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:52.368733 24516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:52.372789 24516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:52.373047 24516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:52.375756 24516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:52.375859 24516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:52.376008 24516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:52.376169 24516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 10 06:49:52 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344592, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:52.396888 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.376166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:38059 (local address 127.22.11.190:40713)
0810 06:49:52.376607 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:52.376632 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:52.377748 (+ 1116us) server_negotiation.cc:407] Connection header received
0810 06:49:52.379736 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:52.379742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:52.379833 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:52.379960 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:52.381802 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.382762 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.385159 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.385612 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.385740 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.390444 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:52.390451 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:52.390456 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:52.390509 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:52.393475 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.394391 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.394397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.394401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.394585 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.395091 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.395097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.395100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.395459 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:52.395599 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:52.396285 (+ 686us) server_negotiation.cc:299] Negotiation successful
0810 06:49:52.396505 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":226,"threads_started":1}
I20260810 06:49:52.397975 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.376840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40713 (local address 127.22.11.129:38059)
0810 06:49:52.377601 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:52.377639 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:52.379419 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:52.380161 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:52.380169 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:52.380773 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:52.381556 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.381594 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.382910 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.382915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.385008 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.385021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.385904 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.385912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.386206 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.386924 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:52.386957 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:52.390206 (+ 3249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:52.393611 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.393619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.393637 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.394238 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.394711 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.394716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.394720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.394955 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.395604 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:52.395612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:52.395927 (+ 315us) client_negotiation.cc:769] Sending connection context
0810 06:49:52.396151 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 06:49:52.396664 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":316,"threads_started":1}
I20260810 06:49:52.422514 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.407491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:37131 (local address 127.22.11.190:40713)
0810 06:49:52.407698 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:52.407704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:52.408679 (+ 975us) server_negotiation.cc:407] Connection header received
0810 06:49:52.408862 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:52.408866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:52.408927 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:52.409025 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:52.410345 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.411262 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.413132 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.413559 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.413684 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.416022 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:52.416027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:52.416030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:52.416063 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:52.419313 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.420324 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.420330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.420333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.420494 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.421091 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.421096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.421100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.421379 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:52.421523 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:52.421947 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 06:49:52.422166 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:49:52.422619 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.407452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40713 (local address 127.22.11.129:37131)
0810 06:49:52.408547 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:52.408570 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:52.408756 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:52.409085 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:52.409090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:52.409761 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:52.410206 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.410218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.411412 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.411418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.413001 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.413014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.413813 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.413824 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.414020 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.414664 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:52.414689 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:52.415826 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:52.419487 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.419496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.419502 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.420191 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.420730 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.420735 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.420739 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.420935 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.421516 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:52.421521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:52.421698 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 06:49:52.421832 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:49:52.422157 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":688,"threads_started":1}
I20260810 06:49:52.426422 24516 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:34617
I20260810 06:49:52.426447 24754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:34617 every 8 connection(s)
I20260810 06:49:52.427707 24516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:52.429677 24755 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40713
I20260810 06:49:52.429896 24755 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:52.430518 24755 heartbeater.cc:511] Master 127.22.11.190:40713 requested a full tablet report, sending...
I20260810 06:49:52.432138 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24516
I20260810 06:49:52.432343 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:52.432336 24411 ts_manager.cc:194] Registered new tserver with Master: a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617)
I20260810 06:49:52.433029 22574 external_mini_cluster.cc:1428] Setting key a4be16039ca86db9a1249de547565725
I20260810 06:49:52.434015 24411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:38059
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:49:52.473762 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40713
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:52.794310 24763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:52.794569 24763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:52.794672 24763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:52.801697 24763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:52.801795 24763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:52.801911 24763 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:49:52.802004 24763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:49:52.811334 24763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:40713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:52.812425 24763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:52.813683 24763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:52.826012 24775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:52.826133 24772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:52.826165 24771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:52.826924 24763 server_base.cc:1034] running on GCE node
I20260810 06:49:52.827530 24763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:52.828449 24763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:52.829679 24763 hybrid_clock.cc:648] HybridClock initialized: now 1786344592829602 us; error 51 us; skew 500 ppm
I20260810 06:49:52.829963 24763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:52 dist-test-slave-f7th krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344592, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:52.832944 24763 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:49:52.834283 24763 webserver.cc:460] Webserver started at http://127.22.11.130:39375/ using document root <none> and password file <none>
I20260810 06:49:52.834923 24763 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:52.835033 24763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:52.835337 24763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:52.838256 24763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "c26c9400a59b446788a8c7b66bf01cca"
format_stamp: "Formatted at 2026-08-10 06:49:52 on dist-test-slave-f7th"
server_key: "d32def85b985a7857ba12d6c3e013193"
server_key_iv: "00196f2ddc6b72296e8a0b35af4a19d1"
server_key_version: "encryptionkey@0"
I20260810 06:49:52.839016 24763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "c26c9400a59b446788a8c7b66bf01cca"
format_stamp: "Formatted at 2026-08-10 06:49:52 on dist-test-slave-f7th"
server_key: "d32def85b985a7857ba12d6c3e013193"
server_key_iv: "00196f2ddc6b72296e8a0b35af4a19d1"
server_key_version: "encryptionkey@0"
I20260810 06:49:52.843969 24763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:49:52.847316 24782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:52.848606 24763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:49:52.848799 24763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "c26c9400a59b446788a8c7b66bf01cca"
format_stamp: "Formatted at 2026-08-10 06:49:52 on dist-test-slave-f7th"
server_key: "d32def85b985a7857ba12d6c3e013193"
server_key_iv: "00196f2ddc6b72296e8a0b35af4a19d1"
server_key_version: "encryptionkey@0"
I20260810 06:49:52.848984 24763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:52.863888 24763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:52.867923 24763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:52.868148 24763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:52.870791 24763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:52.870893 24763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:52.871040 24763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:52.871129 24763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:49:52 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344592, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:52.892738 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.871377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:48421 (local address 127.22.11.190:40713)
0810 06:49:52.871563 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:52.871569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:52.873018 (+ 1449us) server_negotiation.cc:407] Connection header received
0810 06:49:52.875089 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:52.875096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:52.875176 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:52.875264 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:52.877325 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.878280 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.880685 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.881154 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.881265 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.885698 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:52.885705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:52.885711 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:52.885758 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:52.888910 (+ 3152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.889803 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.889809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.889813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.889937 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.890556 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.890562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.890565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.890902 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:52.891066 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:52.892067 (+ 1001us) server_negotiation.cc:299] Negotiation successful
0810 06:49:52.892318 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:49:52.893805 24798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.872051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40713 (local address 127.22.11.130:48421)
0810 06:49:52.872869 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:52.872911 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:52.874788 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:52.875505 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:52.875517 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:52.876219 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:52.877145 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.877176 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.878429 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.878438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.880511 (+ 2073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.880527 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.881404 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.881411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.881712 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.882422 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:52.882457 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:52.885436 (+ 2979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:52.889030 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.889038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.889053 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.889681 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.890042 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.890051 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.890055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.890379 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.891141 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:52.891147 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:52.891632 (+ 485us) client_negotiation.cc:769] Sending connection context
0810 06:49:52.891916 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 06:49:52.892408 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":367,"threads_started":1}
I20260810 06:49:52.915016 24763 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:43399
I20260810 06:49:52.915056 24983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:43399 every 8 connection(s)
I20260810 06:49:52.916525 24763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:52.918664 24986 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40713
I20260810 06:49:52.918903 24986 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:52.919708 24986 heartbeater.cc:511] Master 127.22.11.190:40713 requested a full tablet report, sending...
I20260810 06:49:52.920063 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24763
I20260810 06:49:52.920264 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:52.920390 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.907023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:49981 (local address 127.22.11.190:40713)
0810 06:49:52.907203 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:52.907208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:52.908053 (+ 845us) server_negotiation.cc:407] Connection header received
0810 06:49:52.908281 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:52.908286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:52.908373 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:52.908478 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:52.909759 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.910499 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.911745 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.912162 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.912266 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.914109 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:52.914115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:52.914120 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:52.914170 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:52.917310 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.918223 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.918227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.918231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.918406 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:52.918861 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:52.918866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:52.918869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:52.919319 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:52.919489 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:52.919880 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 06:49:52.920100 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:49:52.920506 24959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:52.907157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40713 (local address 127.22.11.130:49981)
0810 06:49:52.907941 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:52.907965 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:52.908134 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:52.908557 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:52.908562 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:52.909139 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:52.909620 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.909632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.910637 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.910643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.911613 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:52.911624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:52.912332 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:52.912338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:52.912491 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:52.913095 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:52.913119 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:52.913919 (+ 800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:52.917476 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.917481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.917485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.918069 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.918531 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:52.918549 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:52.918553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:52.918719 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:52.919436 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:52.919439 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:52.919616 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 06:49:52.919759 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 06:49:52.920075 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":431,"threads_started":1}
I20260810 06:49:52.920845 22574 external_mini_cluster.cc:1428] Setting key f907c5af93af8daf518b0746142b1bb9
I20260810 06:49:52.921316 24411 ts_manager.cc:194] Registered new tserver with Master: c26c9400a59b446788a8c7b66bf01cca (127.22.11.130:43399)
I20260810 06:49:52.922289 24411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:48421
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:49:52.961416 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40713
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:53.274295 24994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:53.274583 24994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:53.274731 24994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:53.281600 24994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:53.281703 24994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:53.281822 24994 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:49:53.281929 24994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:49:53.291145 24994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:40713
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:53.292266 24994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:53.293557 24994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:53.305853 25003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:53.305987 25002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:53.305962 25005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:53.306785 24994 server_base.cc:1034] running on GCE node
I20260810 06:49:53.307359 24994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:53.308164 24994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:53.309363 24994 hybrid_clock.cc:648] HybridClock initialized: now 1786344593309299 us; error 36 us; skew 500 ppm
I20260810 06:49:53.309644 24994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:53 dist-test-slave-f7th krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344593, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:53.312513 24994 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:49:53.313846 24994 webserver.cc:460] Webserver started at http://127.22.11.131:40781/ using document root <none> and password file <none>
I20260810 06:49:53.314460 24994 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:53.314558 24994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:53.314893 24994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:53.317647 24994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "73f26b9ec5504627acf6d19211f985e7"
format_stamp: "Formatted at 2026-08-10 06:49:53 on dist-test-slave-f7th"
server_key: "d34eb7e8ebfd57be46bd1713687e1765"
server_key_iv: "224f9e3328a62cd9acc27399708732aa"
server_key_version: "encryptionkey@0"
I20260810 06:49:53.318372 24994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "73f26b9ec5504627acf6d19211f985e7"
format_stamp: "Formatted at 2026-08-10 06:49:53 on dist-test-slave-f7th"
server_key: "d34eb7e8ebfd57be46bd1713687e1765"
server_key_iv: "224f9e3328a62cd9acc27399708732aa"
server_key_version: "encryptionkey@0"
I20260810 06:49:53.323122 24994 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:49:53.326287 25021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:53.327553 24994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:49:53.327692 24994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "73f26b9ec5504627acf6d19211f985e7"
format_stamp: "Formatted at 2026-08-10 06:49:53 on dist-test-slave-f7th"
server_key: "d34eb7e8ebfd57be46bd1713687e1765"
server_key_iv: "224f9e3328a62cd9acc27399708732aa"
server_key_version: "encryptionkey@0"
I20260810 06:49:53.327878 24994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:53.351919 24994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:53.355468 24994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:53.355716 24994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:53.358268 24994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:53.358373 24994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:53.358580 24994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:53.358701 24994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:49:53 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344593, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:53.379731 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.358589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:36483 (local address 127.22.11.190:40713)
0810 06:49:53.358755 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.358760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.360336 (+ 1576us) server_negotiation.cc:407] Connection header received
0810 06:49:53.362425 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.362431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.362514 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.362603 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:53.364488 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.365687 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.368077 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.368756 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.368880 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.373340 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:53.373347 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:53.373352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:53.373393 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:53.376532 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:53.377463 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:53.377469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:53.377473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:53.377605 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:53.378024 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:53.378030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:53.378033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:53.378305 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:53.378431 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.379151 (+ 720us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.379386 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:49:53.380695 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.359259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40713 (local address 127.22.11.131:36483)
0810 06:49:53.360177 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:53.360227 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:53.362169 (+ 1942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:53.362826 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:53.362833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:53.363494 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:53.364267 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.364296 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.365850 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.365856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.367902 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.367915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.368981 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.368989 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.369179 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.369820 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:53.369841 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:53.373101 (+ 3260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:53.376733 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:53.376742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:53.376755 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:53.377349 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:53.377719 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:53.377724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:53.377728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:53.377919 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:53.378437 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:53.378444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:53.378753 (+ 309us) client_negotiation.cc:769] Sending connection context
0810 06:49:53.378980 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 06:49:53.379451 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":662,"thread_start_us":454,"threads_started":1}
I20260810 06:49:53.404268 24994 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:39509
I20260810 06:49:53.404335 25215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:39509 every 8 connection(s)
I20260810 06:49:53.405001 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.391368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:37829 (local address 127.22.11.190:40713)
0810 06:49:53.391506 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.391509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.392127 (+ 618us) server_negotiation.cc:407] Connection header received
0810 06:49:53.392279 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.392284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.392343 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.392438 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:53.393818 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.394719 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.396401 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.396881 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.396987 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.399081 (+ 2094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:53.399086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:53.399089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:53.399116 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:53.402118 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:53.402837 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:53.402842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:53.402846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:53.402967 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:53.403315 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:53.403321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:53.403324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:53.403678 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:53.403808 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.404559 (+ 751us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.404771 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:49:53.404987 25162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.391468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40713 (local address 127.22.11.131:37829)
0810 06:49:53.392030 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:53.392047 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:53.392172 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:53.392604 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:53.392608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:53.393214 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:53.393629 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.393646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.394921 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.394929 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.396283 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.396298 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.397064 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.397070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.397225 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.397804 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:53.397821 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:53.398854 (+ 1033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:53.402233 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:53.402238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:53.402242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:53.402729 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:53.403079 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:53.403084 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:53.403088 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:53.403213 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:53.403789 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:53.403794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:53.403970 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 06:49:53.404129 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 06:49:53.404535 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":305,"threads_started":1}
I20260810 06:49:53.405753 24994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:53.407663 25216 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40713
I20260810 06:49:53.407869 25216 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:53.408478 25216 heartbeater.cc:511] Master 127.22.11.190:40713 requested a full tablet report, sending...
I20260810 06:49:53.409722 24411 ts_manager.cc:194] Registered new tserver with Master: 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131:39509)
I20260810 06:49:53.410458 24411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:36483
I20260810 06:49:53.415897 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24994
I20260810 06:49:53.416076 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:53.416486 22574 external_mini_cluster.cc:1428] Setting key f9649dc2c1d77d946c973d3942543d4f
I20260810 06:49:53.419092 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:49:53.437461 24755 heartbeater.cc:503] Master 127.22.11.190:40713 was elected leader, sending a full tablet report...
Aug 10 06:49:53 dist-test-slave-f7th krb5kdc[24331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:49:53 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:53.466388 24554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.453569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.22.11.190:40713)
0810 06:49:53.453769 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.453775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.454023 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:49:53.454208 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.454212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.454274 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.454356 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:53.455778 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.456528 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.457787 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.458210 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.458319 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.462074 (+ 3755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:53.462079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:53.462082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:53.462116 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:53.464340 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:53.464915 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:53.464918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:53.464921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:53.465048 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:53.465347 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:53.465350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:53.465352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:53.465611 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:53.465794 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.465968 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.466130 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:49:53.470327 24411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:53.477048 24411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:53.507575 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.488731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35428 (local address 127.22.11.129:34617)
0810 06:49:53.499064 (+ 10333us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.499073 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.499101 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:49:53.499201 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.499209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.499471 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.501623 (+ 2152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:53.501820 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.503093 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.505320 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.506514 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.506645 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.506873 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.507125 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.507282 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10183,"thread_start_us":10055,"threads_started":1}
I20260810 06:49:53.507644 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.488525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:34617 (local address 127.0.0.1:35428)
0810 06:49:53.490368 (+ 1843us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:53.490406 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:53.490570 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:53.500884 (+ 10314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:53.500895 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:53.500968 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:53.501404 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.501415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.503205 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.503210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.505137 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.505150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.506715 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.506722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.506920 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.506962 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 06:49:53.507084 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:49:53.507257 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1600,"mutex_wait_us":1,"thread_start_us":12066,"threads_started":1}
I20260810 06:49:53.509410 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.490101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:43399 (local address 127.0.0.1:57984)
0810 06:49:53.501202 (+ 11101us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:53.501219 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:53.501333 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:53.502018 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:53.502022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:53.502049 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:53.502599 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.502617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.504395 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.504400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.506770 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.506786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.508453 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.508459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.508631 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.508668 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:49:53.508771 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:49:53.508886 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10972,"mutex_wait_us":505,"thread_start_us":10325,"threads_started":1}
I20260810 06:49:53.509560 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.489217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.22.11.131:39509)
0810 06:49:53.501635 (+ 12418us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.501641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.501666 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:49:53.501745 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.501751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.501896 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.502461 (+ 565us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:53.502895 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.504470 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.506987 (+ 2517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.508537 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.508692 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.508900 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.509116 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.509263 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12280,"thread_start_us":12064,"threads_started":1}
I20260810 06:49:53.510200 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.490446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57984 (local address 127.22.11.130:43399)
0810 06:49:53.500442 (+ 9996us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.500451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.501469 (+ 1018us) server_negotiation.cc:407] Connection header received
0810 06:49:53.501577 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.501584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.501797 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.501993 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:53.502793 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.504238 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.506964 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.508246 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.508385 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.508591 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.509777 (+ 1186us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.509937 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9840,"thread_start_us":9729,"threads_started":1}
I20260810 06:49:53.509563 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.488666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39509 (local address 127.0.0.1:58450)
0810 06:49:53.500322 (+ 11656us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:53.500342 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:53.500447 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:53.502086 (+ 1639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:53.502090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:53.502115 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:53.502677 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.502693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.504657 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.504663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.506787 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.506797 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.508764 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.508770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.508928 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.508959 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:49:53.509043 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:49:53.509164 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11542,"mutex_wait_us":2,"spinlock_wait_cycles":1280,"thread_start_us":2115,"threads_started":1}
I20260810 06:49:53.511590 24632 tablet_service.cc:1467] Processing CreateTablet for tablet 80a54e878a3c4a4c825d987b1fb0e9e6 (DEFAULT_TABLE table=test-table [id=28575bf1c0cd4b7c98173461557b6f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:53.512847 24632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a54e878a3c4a4c825d987b1fb0e9e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:53.513836 25096 tablet_service.cc:1467] Processing CreateTablet for tablet 80a54e878a3c4a4c825d987b1fb0e9e6 (DEFAULT_TABLE table=test-table [id=28575bf1c0cd4b7c98173461557b6f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:53.513837 24860 tablet_service.cc:1467] Processing CreateTablet for tablet 80a54e878a3c4a4c825d987b1fb0e9e6 (DEFAULT_TABLE table=test-table [id=28575bf1c0cd4b7c98173461557b6f25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:53.515142 24860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a54e878a3c4a4c825d987b1fb0e9e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:53.515143 25096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80a54e878a3c4a4c825d987b1fb0e9e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:53.523427 25252 tablet_bootstrap.cc:492] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Bootstrap starting.
I20260810 06:49:53.526638 25252 tablet_bootstrap.cc:654] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:53.528891 25253 tablet_bootstrap.cc:492] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Bootstrap starting.
I20260810 06:49:53.528952 25252 log.cc:824] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:53.530351 25254 tablet_bootstrap.cc:492] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Bootstrap starting.
I20260810 06:49:53.534380 25252 tablet_bootstrap.cc:492] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: No bootstrap required, opened a new log
I20260810 06:49:53.534693 25252 ts_tablet_manager.cc:1397] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260810 06:49:53.535200 25253 tablet_bootstrap.cc:654] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:53.536784 25254 tablet_bootstrap.cc:654] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:53.538051 25253 log.cc:824] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:53.538647 25254 log.cc:824] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:53.542328 25254 tablet_bootstrap.cc:492] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: No bootstrap required, opened a new log
I20260810 06:49:53.542564 25254 ts_tablet_manager.cc:1397] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.003s
I20260810 06:49:53.544116 25253 tablet_bootstrap.cc:492] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: No bootstrap required, opened a new log
I20260810 06:49:53.544600 25253 ts_tablet_manager.cc:1397] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260810 06:49:53.549749 25252 raft_consensus.cc:348] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.550554 25252 raft_consensus.cc:374] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:53.550742 25252 raft_consensus.cc:729] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7172c12d80d4df48ec0eee075fa0ab0, State: Initialized, Role: FOLLOWER
I20260810 06:49:53.551543 25252 consensus_queue.cc:260] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.553011 25254 raft_consensus.cc:348] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.553699 25254 raft_consensus.cc:374] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:53.553932 25254 raft_consensus.cc:729] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c26c9400a59b446788a8c7b66bf01cca, State: Initialized, Role: FOLLOWER
I20260810 06:49:53.554544 25254 consensus_queue.cc:260] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.554773 25252 ts_tablet_manager.cc:1428] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Time spent starting tablet: real 0.020s user 0.021s sys 0.000s
I20260810 06:49:53.556139 24986 heartbeater.cc:503] Master 127.22.11.190:40713 was elected leader, sending a full tablet report...
I20260810 06:49:53.556874 25254 ts_tablet_manager.cc:1428] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Time spent starting tablet: real 0.014s user 0.014s sys 0.002s
I20260810 06:49:53.561506 25253 raft_consensus.cc:348] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.562003 25253 raft_consensus.cc:374] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:53.562189 25253 raft_consensus.cc:729] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f26b9ec5504627acf6d19211f985e7, State: Initialized, Role: FOLLOWER
I20260810 06:49:53.562785 25253 consensus_queue.cc:260] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.564366 25216 heartbeater.cc:503] Master 127.22.11.190:40713 was elected leader, sending a full tablet report...
I20260810 06:49:53.564906 25253 ts_tablet_manager.cc:1428] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Time spent starting tablet: real 0.020s user 0.007s sys 0.011s
W20260810 06:49:53.657485 25217 tablet.cc:2369] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:53.668582 24987 tablet.cc:2369] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:49:53.680029 24757 tablet.cc:2369] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:53.837879 25264 raft_consensus.cc:482] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:53.838475 25264 raft_consensus.cc:504] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.840425 25264 leader_election.cc:288] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617), c26c9400a59b446788a8c7b66bf01cca (127.22.11.130:43399)
I20260810 06:49:53.849691 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.841723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:43399 (local address 127.22.11.131:48511)
0810 06:49:53.841928 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:53.841947 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:53.842081 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:53.842500 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:53.842506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:53.842539 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:53.843065 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.843081 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.844518 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.844521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.847113 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.847129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.848726 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.848733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.849093 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.849132 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 06:49:53.849212 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:49:53.849306 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":1}
I20260810 06:49:53.849877 25248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.841353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:48511 (local address 127.22.11.130:43399)
0810 06:49:53.841615 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.841620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.842038 (+ 418us) server_negotiation.cc:407] Connection header received
0810 06:49:53.842238 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.842245 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.842336 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.842440 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:53.843221 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.844392 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.847293 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.848558 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.848781 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.848863 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.849271 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.849408 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:49:53.851392 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.840913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:42059 (local address 127.22.11.129:34617)
0810 06:49:53.841182 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.841189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.843188 (+ 1999us) server_negotiation.cc:407] Connection header received
0810 06:49:53.843383 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.843389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.843486 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.843607 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:53.844506 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.845571 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.848453 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.850099 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.850285 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.850367 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.850938 (+ 571us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.851081 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:49:53.851840 25265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.841746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:34617 (local address 127.22.11.131:42059)
0810 06:49:53.843072 (+ 1326us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:53.843090 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:53.843251 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:53.843607 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:53.843610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:53.843628 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:53.844325 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.844336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.845723 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.845728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.848293 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:53.848305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.850483 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.850491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:53.850770 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.850803 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:49:53.850903 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:49:53.851037 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1173,"mutex_wait_us":18,"thread_start_us":855,"threads_started":1}
I20260810 06:49:53.851727 24897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a54e878a3c4a4c825d987b1fb0e9e6" candidate_uuid: "73f26b9ec5504627acf6d19211f985e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c26c9400a59b446788a8c7b66bf01cca" is_pre_election: true
I20260810 06:49:53.852761 24897 raft_consensus.cc:2454] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73f26b9ec5504627acf6d19211f985e7 in term 0.
I20260810 06:49:53.853471 24667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a54e878a3c4a4c825d987b1fb0e9e6" candidate_uuid: "73f26b9ec5504627acf6d19211f985e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" is_pre_election: true
I20260810 06:49:53.854126 25025 leader_election.cc:302] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73f26b9ec5504627acf6d19211f985e7, c26c9400a59b446788a8c7b66bf01cca; no voters:
I20260810 06:49:53.854516 24667 raft_consensus.cc:2454] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73f26b9ec5504627acf6d19211f985e7 in term 0.
I20260810 06:49:53.854737 25264 raft_consensus.cc:2781] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:53.855069 25264 raft_consensus.cc:482] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:53.855273 25264 raft_consensus.cc:3037] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:53.858824 25264 raft_consensus.cc:504] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.859661 25264 leader_election.cc:288] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [CANDIDATE]: Term 1 election: Requested vote from peers a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617), c26c9400a59b446788a8c7b66bf01cca (127.22.11.130:43399)
I20260810 06:49:53.860102 24667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a54e878a3c4a4c825d987b1fb0e9e6" candidate_uuid: "73f26b9ec5504627acf6d19211f985e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7172c12d80d4df48ec0eee075fa0ab0"
I20260810 06:49:53.860329 24897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80a54e878a3c4a4c825d987b1fb0e9e6" candidate_uuid: "73f26b9ec5504627acf6d19211f985e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c26c9400a59b446788a8c7b66bf01cca"
I20260810 06:49:53.860400 24667 raft_consensus.cc:3037] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:53.860641 24897 raft_consensus.cc:3037] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:53.864800 24897 raft_consensus.cc:2454] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73f26b9ec5504627acf6d19211f985e7 in term 1.
I20260810 06:49:53.865126 24667 raft_consensus.cc:2454] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73f26b9ec5504627acf6d19211f985e7 in term 1.
I20260810 06:49:53.865437 25025 leader_election.cc:302] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73f26b9ec5504627acf6d19211f985e7, c26c9400a59b446788a8c7b66bf01cca; no voters:
I20260810 06:49:53.865837 25264 raft_consensus.cc:2781] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:53.866750 25264 raft_consensus.cc:686] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 1 LEADER]: Becoming Leader. State: Replica: 73f26b9ec5504627acf6d19211f985e7, State: Running, Role: LEADER
I20260810 06:49:53.867408 25264 consensus_queue.cc:237] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } }
I20260810 06:49:53.873103 24411 catalog_manager.cc:5495] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131). New cstate: current_term: 1 leader_uuid: "73f26b9ec5504627acf6d19211f985e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } health_report { overall_health: UNKNOWN } } }
I20260810 06:49:53.964332 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:53.958169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.22.11.131:39509)
0810 06:49:53.958329 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:53.958333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:53.958632 (+ 299us) server_negotiation.cc:407] Connection header received
0810 06:49:53.958992 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:53.958996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:53.959077 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:53.959190 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:53.959649 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.960561 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.962058 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:53.962637 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:53.962730 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:53.963285 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:53.963814 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:53.963914 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:53.963989 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:49:53.964055 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:49:53.977187 24667 raft_consensus.cc:1261] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Refusing update from remote peer 73f26b9ec5504627acf6d19211f985e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:53.978370 25269 consensus_queue.cc:1035] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:53.979457 24897 raft_consensus.cc:1261] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Refusing update from remote peer 73f26b9ec5504627acf6d19211f985e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:53.980398 25269 consensus_queue.cc:1035] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:53.989933 25273 mvcc.cc:205] Tried to move back new op lower bound from 7316867456904257536 to 7316867456485478400. Current Snapshot: MvccSnapshot[applied={T|T < 7316867456904257536}]
I20260810 06:49:53.992280 25274 mvcc.cc:205] Tried to move back new op lower bound from 7316867456904257536 to 7316867456485478400. Current Snapshot: MvccSnapshot[applied={T|T < 7316867456904257536}]
I20260810 06:49:54.018783 24408 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:54.019239 24408 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:54.023547 24408 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc: Sending DeleteTablet for 3 replicas of tablet 80a54e878a3c4a4c825d987b1fb0e9e6
I20260810 06:49:54.025043 24632 tablet_service.cc:1514] Processing DeleteTablet for tablet 80a54e878a3c4a4c825d987b1fb0e9e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:54 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:35428
I20260810 06:49:54.025271 24860 tablet_service.cc:1514] Processing DeleteTablet for tablet 80a54e878a3c4a4c825d987b1fb0e9e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:54 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:57984
I20260810 06:49:54.025056 25096 tablet_service.cc:1514] Processing DeleteTablet for tablet 80a54e878a3c4a4c825d987b1fb0e9e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:54 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:58450
I20260810 06:49:54.028082 24408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:54.028916 24408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:54.040282 24859 tablet_service.cc:1467] Processing CreateTablet for tablet 38dcf17dc9ce4e2d8e93c420b41401cb (DEFAULT_TABLE table=test-table [id=8b26a16974804f5a8395647be79ef2da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:54.040253 24631 tablet_service.cc:1467] Processing CreateTablet for tablet 38dcf17dc9ce4e2d8e93c420b41401cb (DEFAULT_TABLE table=test-table [id=8b26a16974804f5a8395647be79ef2da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:54.040918 25095 tablet_service.cc:1467] Processing CreateTablet for tablet 38dcf17dc9ce4e2d8e93c420b41401cb (DEFAULT_TABLE table=test-table [id=8b26a16974804f5a8395647be79ef2da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:54.041177 24631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38dcf17dc9ce4e2d8e93c420b41401cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:54.041194 24859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38dcf17dc9ce4e2d8e93c420b41401cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:54.041657 25095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38dcf17dc9ce4e2d8e93c420b41401cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:54.042487 25292 tablet_replica.cc:330] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: stopping tablet replica
I20260810 06:49:54.048691 25252 tablet_bootstrap.cc:492] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: Bootstrap starting.
I20260810 06:49:54.043082 25292 raft_consensus.cc:2229] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:49:54.050168 25253 tablet_bootstrap.cc:492] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: Bootstrap starting.
I20260810 06:49:54.050545 25252 tablet_bootstrap.cc:654] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:54.051862 25253 tablet_bootstrap.cc:654] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:54.053318 25254 tablet_bootstrap.cc:492] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: Bootstrap starting.
I20260810 06:49:54.055830 25254 tablet_bootstrap.cc:654] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:54.058610 25292 raft_consensus.cc:2258] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:54.064352 25292 ts_tablet_manager.cc:1905] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:54.064574 25254 tablet_bootstrap.cc:492] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: No bootstrap required, opened a new log
I20260810 06:49:54.064827 25254 ts_tablet_manager.cc:1397] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.002s
I20260810 06:49:54.067214 25254 raft_consensus.cc:348] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.067538 25254 raft_consensus.cc:374] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:54.067824 25254 raft_consensus.cc:729] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c26c9400a59b446788a8c7b66bf01cca, State: Initialized, Role: FOLLOWER
I20260810 06:49:54.068524 25290 tablet_replica.cc:330] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: stopping tablet replica
I20260810 06:49:54.068866 25287 tablet_replica.cc:330] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: stopping tablet replica
I20260810 06:49:54.069177 25252 tablet_bootstrap.cc:492] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: No bootstrap required, opened a new log
I20260810 06:49:54.068742 25254 consensus_queue.cc:260] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.069320 25287 raft_consensus.cc:2229] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:54.069339 25252 ts_tablet_manager.cc:1397] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.009s
I20260810 06:49:54.069861 25287 raft_consensus.cc:2258] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:54.070982 25253 tablet_bootstrap.cc:492] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: No bootstrap required, opened a new log
I20260810 06:49:54.071190 25253 ts_tablet_manager.cc:1397] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.005s
I20260810 06:49:54.071954 25254 ts_tablet_manager.cc:1428] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260810 06:49:54.072259 25253 raft_consensus.cc:348] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.072213 25290 raft_consensus.cc:2229] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:54.072449 25252 raft_consensus.cc:348] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.072608 25253 raft_consensus.cc:374] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:54.072862 25252 raft_consensus.cc:374] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:54.072860 25253 raft_consensus.cc:729] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f26b9ec5504627acf6d19211f985e7, State: Initialized, Role: FOLLOWER
I20260810 06:49:54.073127 25290 raft_consensus.cc:2258] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:54.073227 25253 consensus_queue.cc:260] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.073360 25252 raft_consensus.cc:729] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7172c12d80d4df48ec0eee075fa0ab0, State: Initialized, Role: FOLLOWER
I20260810 06:49:54.073699 25252 consensus_queue.cc:260] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.075732 25252 ts_tablet_manager.cc:1428] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:49:54.076771 25290 ts_tablet_manager.cc:1905] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:54.076901 25253 ts_tablet_manager.cc:1428] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260810 06:49:54.082041 25287 ts_tablet_manager.cc:1905] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:54.084761 25292 ts_tablet_manager.cc:1918] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:54.085000 25292 log.cc:1196] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/80a54e878a3c4a4c825d987b1fb0e9e6
I20260810 06:49:54.085712 25292 ts_tablet_manager.cc:1939] T 80a54e878a3c4a4c825d987b1fb0e9e6 P 73f26b9ec5504627acf6d19211f985e7: Deleting consensus metadata
I20260810 06:49:54.086763 25263 raft_consensus.cc:482] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:54.086977 25263 raft_consensus.cc:504] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.087785 24383 catalog_manager.cc:4841] TS 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131:39509): tablet 80a54e878a3c4a4c825d987b1fb0e9e6 (table test-table [id=28575bf1c0cd4b7c98173461557b6f25]) successfully deleted
I20260810 06:49:54.089772 25290 ts_tablet_manager.cc:1918] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:54.089955 25290 log.cc:1196] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/80a54e878a3c4a4c825d987b1fb0e9e6
I20260810 06:49:54.090559 25290 ts_tablet_manager.cc:1939] T 80a54e878a3c4a4c825d987b1fb0e9e6 P c26c9400a59b446788a8c7b66bf01cca: Deleting consensus metadata
I20260810 06:49:54.091761 25287 ts_tablet_manager.cc:1918] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:49:54.092036 25287 log.cc:1196] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/80a54e878a3c4a4c825d987b1fb0e9e6
I20260810 06:49:54.092492 24384 catalog_manager.cc:4841] TS c26c9400a59b446788a8c7b66bf01cca (127.22.11.130:43399): tablet 80a54e878a3c4a4c825d987b1fb0e9e6 (table test-table [id=28575bf1c0cd4b7c98173461557b6f25]) successfully deleted
I20260810 06:49:54.092734 25287 ts_tablet_manager.cc:1939] T 80a54e878a3c4a4c825d987b1fb0e9e6 P a7172c12d80d4df48ec0eee075fa0ab0: Deleting consensus metadata
I20260810 06:49:54.089063 25263 leader_election.cc:288] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617), 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131:39509)
I20260810 06:49:54.094858 24386 catalog_manager.cc:4841] TS a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617): tablet 80a54e878a3c4a4c825d987b1fb0e9e6 (table test-table [id=28575bf1c0cd4b7c98173461557b6f25]) successfully deleted
I20260810 06:49:54.102689 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.091362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:36141 (local address 127.22.11.129:34617)
0810 06:49:54.091527 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.091532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.094643 (+ 3111us) server_negotiation.cc:407] Connection header received
0810 06:49:54.094793 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.094798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.094939 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.095036 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:54.096214 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.097227 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.099821 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.101499 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.101671 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.101750 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.102273 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.102420 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:49:54.102998 25299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.089648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39509 (local address 127.22.11.130:39189)
0810 06:49:54.093514 (+ 3866us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:54.093534 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:54.093702 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:54.095308 (+ 1606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:54.095312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:54.095334 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:54.096287 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.096303 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.098219 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.098224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.100379 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.100391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.102027 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.102034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.102199 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.102228 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:49:54.102306 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:49:54.102435 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3742,"mutex_wait_us":1,"thread_start_us":5253,"threads_started":1}
I20260810 06:49:54.103302 25247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.090150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:39189 (local address 127.22.11.131:39509)
0810 06:49:54.090320 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.090325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.094090 (+ 3765us) server_negotiation.cc:407] Connection header received
0810 06:49:54.094178 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.094184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.094320 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.094423 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:54.096482 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.098070 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.100546 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.101644 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.101790 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.101859 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.102345 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.102487 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:49:54.103506 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.090845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:34617 (local address 127.22.11.130:36141)
0810 06:49:54.094535 (+ 3690us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:54.094559 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:54.094678 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:54.095269 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:54.095275 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:54.095307 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:54.096008 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.096025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.097355 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.097360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.099666 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.099682 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.101825 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.101833 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.102077 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.102112 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:49:54.102860 (+ 748us) client_negotiation.cc:240] Negotiation successful
0810 06:49:54.102999 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3545,"mutex_wait_us":21,"thread_start_us":4150,"threads_started":1}
I20260810 06:49:54.104337 24661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38dcf17dc9ce4e2d8e93c420b41401cb" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" is_pre_election: true
I20260810 06:49:54.104256 25135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38dcf17dc9ce4e2d8e93c420b41401cb" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f26b9ec5504627acf6d19211f985e7" is_pre_election: true
I20260810 06:49:54.104691 24661 raft_consensus.cc:2454] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 0.
I20260810 06:49:54.105005 25135 raft_consensus.cc:2454] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 0.
I20260810 06:49:54.105276 24789 leader_election.cc:302] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7172c12d80d4df48ec0eee075fa0ab0, c26c9400a59b446788a8c7b66bf01cca; no voters:
I20260810 06:49:54.105712 25263 raft_consensus.cc:2781] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:54.105924 25263 raft_consensus.cc:482] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:54.106081 25263 raft_consensus.cc:3037] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:54.109921 25263 raft_consensus.cc:504] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.110644 25263 leader_election.cc:288] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 election: Requested vote from peers a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617), 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131:39509)
I20260810 06:49:54.111035 24661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38dcf17dc9ce4e2d8e93c420b41401cb" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7172c12d80d4df48ec0eee075fa0ab0"
I20260810 06:49:54.111222 24661 raft_consensus.cc:3037] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:54.111232 25135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38dcf17dc9ce4e2d8e93c420b41401cb" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f26b9ec5504627acf6d19211f985e7"
I20260810 06:49:54.111435 25135 raft_consensus.cc:3037] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:54.113291 24661 raft_consensus.cc:2454] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 1.
I20260810 06:49:54.113628 24789 leader_election.cc:302] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7172c12d80d4df48ec0eee075fa0ab0, c26c9400a59b446788a8c7b66bf01cca; no voters:
I20260810 06:49:54.113970 25263 raft_consensus.cc:2781] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:54.114866 25263 raft_consensus.cc:686] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 1 LEADER]: Becoming Leader. State: Replica: c26c9400a59b446788a8c7b66bf01cca, State: Running, Role: LEADER
I20260810 06:49:54.115445 25135 raft_consensus.cc:2454] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 1.
I20260810 06:49:54.115422 25263 consensus_queue.cc:237] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.119352 24411 catalog_manager.cc:5495] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca reported cstate change: term changed from 0 to 1, leader changed from <none> to c26c9400a59b446788a8c7b66bf01cca (127.22.11.130). New cstate: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } health_report { overall_health: UNKNOWN } } }
Aug 10 06:49:54 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344591, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:54.182341 25312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.163076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.22.11.190:40713)
0810 06:49:54.164235 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.164239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.164270 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:49:54.164316 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.164319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.164373 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.164463 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:54.165901 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.166911 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.168415 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.169017 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.169127 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.176012 (+ 6885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:54.176018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:54.176021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:54.176050 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:54.179074 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.179896 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.179901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.179903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.180078 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.180473 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.180476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.180478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.180899 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:54.181049 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.181616 (+ 567us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.181914 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":120,"server-negotiator.queue_time_us":1034,"thread_start_us":877,"threads_started":1}
I20260810 06:49:54.182514 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.162935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40713 (local address 127.0.0.1:38440)
0810 06:49:54.163676 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:54.163700 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:54.163851 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:54.164517 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:54.164521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:54.165328 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:54.165677 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.165689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.167085 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.167089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.168281 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.168294 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.169200 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.169206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.169421 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.172931 (+ 3510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:54.172948 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:54.175746 (+ 2798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:54.179221 (+ 3475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:54.179234 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:54.179242 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:54.179792 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:54.180179 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:54.180184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:54.180188 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:54.180372 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:54.181052 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:54.181060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:54.181280 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 06:49:54.181530 (+ 250us) client_negotiation.cc:240] Negotiation successful
0810 06:49:54.181788 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":431,"threads_started":1}
I20260810 06:49:54.188218 24411 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:38440:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:49:54.189572 24411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:54.196318 24632 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca8334e75c440f6a93218de71b443c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=45750368ce264f5aab3a7f15b048eef2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:49:54.196532 24860 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca8334e75c440f6a93218de71b443c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=45750368ce264f5aab3a7f15b048eef2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:49:54.196995 24632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca8334e75c440f6a93218de71b443c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:54.196882 25095 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca8334e75c440f6a93218de71b443c8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=45750368ce264f5aab3a7f15b048eef2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:49:54.197186 24860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca8334e75c440f6a93218de71b443c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:54.197497 25095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca8334e75c440f6a93218de71b443c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:54.205696 25253 tablet_bootstrap.cc:492] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: Bootstrap starting.
I20260810 06:49:54.206619 25254 tablet_bootstrap.cc:492] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: Bootstrap starting.
I20260810 06:49:54.208523 25252 tablet_bootstrap.cc:492] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: Bootstrap starting.
I20260810 06:49:54.208896 25254 tablet_bootstrap.cc:654] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:54.209432 25253 tablet_bootstrap.cc:654] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:54.211196 25252 tablet_bootstrap.cc:654] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:54.220373 25252 tablet_bootstrap.cc:492] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: No bootstrap required, opened a new log
I20260810 06:49:54.220574 25252 ts_tablet_manager.cc:1397] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260810 06:49:54.221367 25254 tablet_bootstrap.cc:492] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: No bootstrap required, opened a new log
I20260810 06:49:54.221355 25252 raft_consensus.cc:348] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.221536 25254 ts_tablet_manager.cc:1397] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.012s
I20260810 06:49:54.221587 25252 raft_consensus.cc:374] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:54.221673 25252 raft_consensus.cc:729] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7172c12d80d4df48ec0eee075fa0ab0, State: Initialized, Role: FOLLOWER
I20260810 06:49:54.221952 25252 consensus_queue.cc:260] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.222322 25254 raft_consensus.cc:348] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.222568 25254 raft_consensus.cc:374] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:54.222715 25254 raft_consensus.cc:729] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c26c9400a59b446788a8c7b66bf01cca, State: Initialized, Role: FOLLOWER
I20260810 06:49:54.222390 25262 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.222867 25262 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:54.223340 25252 ts_tablet_manager.cc:1428] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:49:54.224116 25254 consensus_queue.cc:260] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.224888 25254 ts_tablet_manager.cc:1428] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260810 06:49:54.224782 25263 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.225134 25263 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:54.225363 25253 tablet_bootstrap.cc:492] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: No bootstrap required, opened a new log
I20260810 06:49:54.225524 25253 ts_tablet_manager.cc:1397] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.010s
I20260810 06:49:54.226576 25253 raft_consensus.cc:348] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.226951 25253 raft_consensus.cc:374] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:54.227065 25253 raft_consensus.cc:729] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f26b9ec5504627acf6d19211f985e7, State: Initialized, Role: FOLLOWER
I20260810 06:49:54.227367 25253 consensus_queue.cc:260] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.228700 25253 ts_tablet_manager.cc:1428] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260810 06:49:54.228149 25282 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.229159 25282 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:54.445681 25263 raft_consensus.cc:482] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:54.445945 25263 raft_consensus.cc:504] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.447075 25263 leader_election.cc:288] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617), 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131:39509)
I20260810 06:49:54.447623 25135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca8334e75c440f6a93218de71b443c8" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f26b9ec5504627acf6d19211f985e7" is_pre_election: true
I20260810 06:49:54.447866 25135 raft_consensus.cc:2454] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 0.
I20260810 06:49:54.447978 24661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca8334e75c440f6a93218de71b443c8" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" is_pre_election: true
I20260810 06:49:54.448342 24661 raft_consensus.cc:2454] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 0.
I20260810 06:49:54.448360 24783 leader_election.cc:302] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73f26b9ec5504627acf6d19211f985e7, c26c9400a59b446788a8c7b66bf01cca; no voters:
I20260810 06:49:54.449082 25263 raft_consensus.cc:2781] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:54.449216 25263 raft_consensus.cc:482] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:54.449395 25263 raft_consensus.cc:3037] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:54.452350 25263 raft_consensus.cc:504] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.453058 25263 leader_election.cc:288] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 election: Requested vote from peers a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617), 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131:39509)
I20260810 06:49:54.453462 24661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca8334e75c440f6a93218de71b443c8" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7172c12d80d4df48ec0eee075fa0ab0"
I20260810 06:49:54.453593 25135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca8334e75c440f6a93218de71b443c8" candidate_uuid: "c26c9400a59b446788a8c7b66bf01cca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73f26b9ec5504627acf6d19211f985e7"
I20260810 06:49:54.453847 24661 raft_consensus.cc:3037] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:54.453940 25135 raft_consensus.cc:3037] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:54.456856 25135 raft_consensus.cc:2454] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 1.
I20260810 06:49:54.457301 24783 leader_election.cc:302] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73f26b9ec5504627acf6d19211f985e7, c26c9400a59b446788a8c7b66bf01cca; no voters:
I20260810 06:49:54.457561 25263 raft_consensus.cc:2781] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:54.457649 24661 raft_consensus.cc:2454] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c26c9400a59b446788a8c7b66bf01cca in term 1.
I20260810 06:49:54.458173 25263 raft_consensus.cc:686] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [term 1 LEADER]: Becoming Leader. State: Replica: c26c9400a59b446788a8c7b66bf01cca, State: Running, Role: LEADER
I20260810 06:49:54.458499 25263 consensus_queue.cc:237] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } }
I20260810 06:49:54.460777 25303 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: TxnStatusTablet state changed. Reason: New leader c26c9400a59b446788a8c7b66bf01cca. Latest consensus state: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.461458 25303 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:49:54.462800 25324 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:49:54.462626 24408 catalog_manager.cc:5495] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca reported cstate change: term changed from 0 to 1, leader changed from <none> to c26c9400a59b446788a8c7b66bf01cca (127.22.11.130). New cstate: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } health_report { overall_health: UNKNOWN } } }
I20260810 06:49:54.463201 25324 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:49:54.518561 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.508542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:43399 (local address 127.0.0.1:57990)
0810 06:49:54.509087 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:54.509109 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:54.509249 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:54.510739 (+ 1490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:54.510745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:54.510759 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:49:54.510989 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.511005 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.512659 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.512665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.514525 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.514553 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.515866 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.515873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.516326 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.516407 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:49:54.517676 (+ 1269us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:49:54.517718 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 06:49:54.517854 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 06:49:54.518050 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307}
I20260810 06:49:54.518573 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.508329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.22.11.130:43399)
0810 06:49:54.510184 (+ 1855us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.510190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.510209 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:49:54.510310 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.510316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.510504 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.510683 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:54.511161 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.512477 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.514705 (+ 2228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.515683 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.515820 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.516661 (+ 841us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:54.517509 (+ 848us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:54.517663 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.517898 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.517984 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1718,"thread_start_us":1435,"threads_started":1}
I20260810 06:49:54.526381 25135 raft_consensus.cc:1261] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Refusing update from remote peer c26c9400a59b446788a8c7b66bf01cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:54.526368 24661 raft_consensus.cc:1261] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Refusing update from remote peer c26c9400a59b446788a8c7b66bf01cca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:54.527850 25263 consensus_queue.cc:1035] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:54.528697 25303 consensus_queue.cc:1035] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:54.536293 25303 consensus_queue.cc:1035] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260810 06:49:54.557098 25282 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: TxnStatusTablet state changed. Reason: New leader c26c9400a59b446788a8c7b66bf01cca. Latest consensus state: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.557449 25282 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:54.561685 25332 mvcc.cc:205] Tried to move back new op lower bound from 7316867459169054720 to 7316867458904727552. Current Snapshot: MvccSnapshot[applied={T|T < 7316867459169054720}]
I20260810 06:49:54.564922 25282 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.565232 25282 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P 73f26b9ec5504627acf6d19211f985e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:54.565122 25263 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.565424 25263 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:49:54.580550 25303 consensus_queue.cc:1035] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [LEADER]: Connected to new peer: Peer: permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:54.604121 25262 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: TxnStatusTablet state changed. Reason: New leader c26c9400a59b446788a8c7b66bf01cca. Latest consensus state: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.604423 25262 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:54.605939 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.592275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.22.11.130:43399)
0810 06:49:54.592472 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.592478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.597316 (+ 4838us) server_negotiation.cc:407] Connection header received
0810 06:49:54.597546 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.597552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.597641 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.597742 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:54.598495 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.599855 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.602733 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.604272 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.604388 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.605076 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:54.605219 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:54.605288 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.605564 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.605652 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:49:54.609963 25263 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.610275 25263 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P c26c9400a59b446788a8c7b66bf01cca: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:49:54.644898 25262 tablet_replica.cc:439] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c26c9400a59b446788a8c7b66bf01cca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7172c12d80d4df48ec0eee075fa0ab0" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 34617 } } peers { permanent_uuid: "c26c9400a59b446788a8c7b66bf01cca" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43399 } } peers { permanent_uuid: "73f26b9ec5504627acf6d19211f985e7" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39509 } } }
I20260810 06:49:54.645220 25262 tablet_replica.cc:442] T 6ca8334e75c440f6a93218de71b443c8 P a7172c12d80d4df48ec0eee075fa0ab0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:54.655507 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.640147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:57561 (local address 127.22.11.130:43399)
0810 06:49:54.640391 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.640400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.642140 (+ 1740us) server_negotiation.cc:407] Connection header received
0810 06:49:54.644205 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.644210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.644388 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.644455 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:54.647756 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.649557 (+ 1801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.652437 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.653694 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.653848 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.654552 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:54.654689 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:54.654790 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.654944 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.655039 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:49:54.655803 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.640308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:43399 (local address 127.22.11.130:57561)
0810 06:49:54.641911 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:54.641932 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:54.644057 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:54.644696 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:54.644700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:54.644713 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:49:54.647500 (+ 2787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.647525 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.649757 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.649763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.652293 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:54.652311 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.653903 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.653911 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:54.654255 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.654339 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:49:54.654806 (+ 467us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:49:54.654819 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 06:49:54.654906 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:49:54.655066 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1360,"thread_start_us":1186,"threads_started":1}
I20260810 06:49:54.736040 24411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:54.736286 24411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:54.739482 24411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c11942ab9e5b4e89b605876a13eaaecc: Sending DeleteTablet for 3 replicas of tablet 38dcf17dc9ce4e2d8e93c420b41401cb
I20260810 06:49:54.740355 24632 tablet_service.cc:1514] Processing DeleteTablet for tablet 38dcf17dc9ce4e2d8e93c420b41401cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:54 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:35428
I20260810 06:49:54.740391 24860 tablet_service.cc:1514] Processing DeleteTablet for tablet 38dcf17dc9ce4e2d8e93c420b41401cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:54 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:57984
I20260810 06:49:54.740870 25095 tablet_service.cc:1514] Processing DeleteTablet for tablet 38dcf17dc9ce4e2d8e93c420b41401cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:54 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:58450
I20260810 06:49:54.751716 25362 tablet_replica.cc:330] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: stopping tablet replica
I20260810 06:49:54.752161 25363 tablet_replica.cc:330] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: stopping tablet replica
I20260810 06:49:54.752485 25363 raft_consensus.cc:2229] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:54.752347 25362 raft_consensus.cc:2229] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:49:54.753070 25363 raft_consensus.cc:2258] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:54.754336 25363 ts_tablet_manager.cc:1905] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:54.754920 25360 tablet_replica.cc:330] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: stopping tablet replica
I20260810 06:49:54.755307 25360 raft_consensus.cc:2229] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:54.755999 25360 raft_consensus.cc:2258] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:54.757299 25360 ts_tablet_manager.cc:1905] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:54.764060 25363 ts_tablet_manager.cc:1918] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:49:54.764341 25363 log.cc:1196] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/38dcf17dc9ce4e2d8e93c420b41401cb
I20260810 06:49:54.764760 25362 raft_consensus.cc:2258] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:54.765245 25363 ts_tablet_manager.cc:1939] T 38dcf17dc9ce4e2d8e93c420b41401cb P 73f26b9ec5504627acf6d19211f985e7: Deleting consensus metadata
I20260810 06:49:54.766008 25362 ts_tablet_manager.cc:1905] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:54.766479 24383 catalog_manager.cc:4841] TS 73f26b9ec5504627acf6d19211f985e7 (127.22.11.131:39509): tablet 38dcf17dc9ce4e2d8e93c420b41401cb (table test-table [id=8b26a16974804f5a8395647be79ef2da]) successfully deleted
I20260810 06:49:54.767635 25360 ts_tablet_manager.cc:1918] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:49:54.767799 25360 log.cc:1196] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/38dcf17dc9ce4e2d8e93c420b41401cb
I20260810 06:49:54.768364 25360 ts_tablet_manager.cc:1939] T 38dcf17dc9ce4e2d8e93c420b41401cb P a7172c12d80d4df48ec0eee075fa0ab0: Deleting consensus metadata
I20260810 06:49:54.769496 24386 catalog_manager.cc:4841] TS a7172c12d80d4df48ec0eee075fa0ab0 (127.22.11.129:34617): tablet 38dcf17dc9ce4e2d8e93c420b41401cb (table test-table [id=8b26a16974804f5a8395647be79ef2da]) successfully deleted
I20260810 06:49:54.772759 25362 ts_tablet_manager.cc:1918] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:49:54.773029 25362 log.cc:1196] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/38dcf17dc9ce4e2d8e93c420b41401cb
I20260810 06:49:54.773856 25362 ts_tablet_manager.cc:1939] T 38dcf17dc9ce4e2d8e93c420b41401cb P c26c9400a59b446788a8c7b66bf01cca: Deleting consensus metadata
I20260810 06:49:54.775009 24384 catalog_manager.cc:4841] TS c26c9400a59b446788a8c7b66bf01cca (127.22.11.130:43399): tablet 38dcf17dc9ce4e2d8e93c420b41401cb (table test-table [id=8b26a16974804f5a8395647be79ef2da]) successfully deleted
Aug 10 06:49:54 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:49:54.787789 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.755013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35442 (local address 127.22.11.129:34617)
0810 06:49:54.764131 (+ 9118us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.764138 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.765120 (+ 982us) server_negotiation.cc:407] Connection header received
0810 06:49:54.765370 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.765375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.765456 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.765573 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:54.768745 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.770244 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.772843 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.773907 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.774047 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.780007 (+ 5960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:54.780022 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:54.780038 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:54.780072 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:54.784440 (+ 4368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.785328 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.785336 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.785338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.785531 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.785957 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.785961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.785963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.786419 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:54.786799 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.787122 (+ 323us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.787369 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5875,"thread_start_us":5714,"threads_started":1}
W20260810 06:49:54.788949 24748 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35442
I20260810 06:49:54.810231 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.792354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38456 (local address 127.22.11.190:40713)
0810 06:49:54.793229 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.793234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.793249 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:49:54.793299 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.793302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.793370 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.793465 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:54.795274 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.796835 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.798608 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.799535 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.799654 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.803234 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:54.803241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:54.803246 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:54.803285 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:54.806988 (+ 3703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.807808 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.807813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.807817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.807974 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.808364 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.808369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.808373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.808763 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:54.809022 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.809442 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.809668 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":524,"threads_started":1}
W20260810 06:49:54.810906 24411 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38456
I20260810 06:49:54.813567 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24516
I20260810 06:49:54.836225 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.815225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.22.11.129:34617)
0810 06:49:54.815407 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.815413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.821320 (+ 5907us) server_negotiation.cc:407] Connection header received
0810 06:49:54.821448 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.821454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.821542 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.821655 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:54.823056 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.824164 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.825486 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.826150 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.826278 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.828898 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:54.828906 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:54.828912 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:54.828960 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:54.832944 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.833703 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.833709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.833713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.833927 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.834388 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.834394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.834397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.834802 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:54.835050 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.835485 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.835749 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 06:49:54.837116 24748 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35446
W20260810 06:49:54.837798 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24516: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:54.838076 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24516
I20260810 06:49:54.865077 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24763
Aug 10 06:49:54 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:49:54.880954 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.865937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.22.11.130:43399)
0810 06:49:54.866169 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.866175 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.866197 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:49:54.866266 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.866271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.866361 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.866486 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:54.867625 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.868982 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.870152 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.870974 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.871113 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.875254 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:54.875267 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:54.875281 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:54.875317 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:54.878145 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.878972 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.878978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.878980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.879141 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.879563 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.879566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.879567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.879904 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:54.880174 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.880460 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.880647 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 06:49:54.881750 24977 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58006
W20260810 06:49:54.882473 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24763: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:54.882664 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24763
I20260810 06:49:54.910471 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24994
Aug 10 06:49:54 dist-test-slave-f7th krb5kdc[24331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344593, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:49:54.924983 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:54.911199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58460 (local address 127.22.11.131:39509)
0810 06:49:54.911641 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:54.911646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:54.911663 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:49:54.911720 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:54.911724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:54.911783 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:54.911882 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:54.913135 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.913813 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.914955 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:54.915363 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:54.915456 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:54.919442 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:54.919458 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:54.919475 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:54.919508 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:54.922284 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.922923 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.922931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.922933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.923071 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:54.923403 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:54.923407 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:54.923409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:54.923728 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:54.924014 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:54.924268 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 06:49:54.924479 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":234,"threads_started":1}
W20260810 06:49:54.925612 25208 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58460
W20260810 06:49:54.926182 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24994: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:49:54.926304 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24994
I20260810 06:49:54.959342 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24360
I20260810 06:49:54.960502 24466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:49:55.074134 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24360
2026-08-10T06:49:55Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5077 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:49:55 dist-test-slave-f7th krb5kdc[25385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:49:55 dist-test-slave-f7th krb5kdc[25385](info): set up 2 sockets
Aug 10 06:49:55 dist-test-slave-f7th krb5kdc[25385](info): commencing operation
krb5kdc: starting...
W20260810 06:49:56.296953 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:49:56 dist-test-slave-f7th krb5kdc[25385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:49:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:49:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:49:56.474944 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40623
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:56.785665 25412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:56.785964 25412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:56.786096 25412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:56.793269 25412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:49:56.793380 25412 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:49:56.793454 25412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:56.793546 25412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:56.793614 25412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:49:56.793700 25412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:49:56.803375 25412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40623
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:56.804580 25412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:56.805908 25412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:56.817056 25418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:56.817143 25421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:56.817104 25423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:56.818303 25412 server_base.cc:1034] running on GCE node
I20260810 06:49:56.818971 25412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:56.820178 25412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:56.821383 25412 hybrid_clock.cc:648] HybridClock initialized: now 1786344596821321 us; error 29 us; skew 500 ppm
I20260810 06:49:56.821668 25412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:56 dist-test-slave-f7th krb5kdc[25385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344596, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:56.824572 25412 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:49:56.825927 25412 webserver.cc:460] Webserver started at http://127.22.11.190:34601/ using document root <none> and password file <none>
I20260810 06:49:56.826553 25412 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:56.826638 25412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:56.826908 25412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:56.829748 25412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "3a851e9f0d9249ca9eb8b61814e78902"
format_stamp: "Formatted at 2026-08-10 06:49:56 on dist-test-slave-f7th"
server_key: "a7fa1b5e56337034614ab81df6ed0c55"
server_key_iv: "5dadac7e9ab068f4b0253090a3b96d69"
server_key_version: "encryptionkey@0"
I20260810 06:49:56.830436 25412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "3a851e9f0d9249ca9eb8b61814e78902"
format_stamp: "Formatted at 2026-08-10 06:49:56 on dist-test-slave-f7th"
server_key: "a7fa1b5e56337034614ab81df6ed0c55"
server_key_iv: "5dadac7e9ab068f4b0253090a3b96d69"
server_key_version: "encryptionkey@0"
I20260810 06:49:56.835181 25412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:49:56.838387 25436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:56.839591 25412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:49:56.839723 25412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "3a851e9f0d9249ca9eb8b61814e78902"
format_stamp: "Formatted at 2026-08-10 06:49:56 on dist-test-slave-f7th"
server_key: "a7fa1b5e56337034614ab81df6ed0c55"
server_key_iv: "5dadac7e9ab068f4b0253090a3b96d69"
server_key_version: "encryptionkey@0"
I20260810 06:49:56.839895 25412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:56.858194 25412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:56.862250 25412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:56.862464 25412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:56.881652 25412 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:40623
I20260810 06:49:56.881682 25547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:40623 every 8 connection(s)
I20260810 06:49:56.882897 25412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:49:56.886466 25549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:56.889111 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25412
I20260810 06:49:56.889320 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:49:56.890151 22574 external_mini_cluster.cc:1428] Setting key 8dd031747c195a1e4b609237dcc7267f
I20260810 06:49:56.898573 25549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Bootstrap starting.
I20260810 06:49:56.902405 25549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:56.903687 25549 log.cc:824] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:49:56 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:56.906457 25549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: No bootstrap required, opened a new log
I20260810 06:49:56.912899 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:56.892169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43636 (local address 127.22.11.190:40623)
0810 06:49:56.892923 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:56.892933 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:56.892971 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:49:56.894612 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:56.894637 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:56.894991 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:56.895307 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:56.897188 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:56.898427 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:56.900355 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:56.901046 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:56.901174 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:56.905887 (+ 4713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:56.905895 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:56.905909 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:56.905943 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:56.908916 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:56.909585 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:56.909591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:56.909597 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:56.909764 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:56.910141 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:56.910145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:56.910147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:56.910693 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:56.910995 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:56.911446 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 06:49:56.911847 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":365,"threads_started":1}
I20260810 06:49:56.914129 25549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a851e9f0d9249ca9eb8b61814e78902" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40623 } }
I20260810 06:49:56.914469 25549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:56.914572 25549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a851e9f0d9249ca9eb8b61814e78902, State: Initialized, Role: FOLLOWER
I20260810 06:49:56.915093 25549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a851e9f0d9249ca9eb8b61814e78902" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40623 } }
I20260810 06:49:56.915323 25549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:49:56.915455 25549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:49:56.915617 25549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:56.917698 25549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a851e9f0d9249ca9eb8b61814e78902" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40623 } }
I20260810 06:49:56.918139 25549 leader_election.cc:302] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a851e9f0d9249ca9eb8b61814e78902; no voters:
I20260810 06:49:56.918655 25549 leader_election.cc:288] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:49:56.918763 25561 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:56.919426 25561 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [term 1 LEADER]: Becoming Leader. State: Replica: 3a851e9f0d9249ca9eb8b61814e78902, State: Running, Role: LEADER
I20260810 06:49:56.919950 25561 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a851e9f0d9249ca9eb8b61814e78902" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40623 } }
I20260810 06:49:56.920454 25549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:49:56.922658 25563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a851e9f0d9249ca9eb8b61814e78902. Latest consensus state: current_term: 1 leader_uuid: "3a851e9f0d9249ca9eb8b61814e78902" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a851e9f0d9249ca9eb8b61814e78902" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40623 } } }
I20260810 06:49:56.922643 25562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a851e9f0d9249ca9eb8b61814e78902" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a851e9f0d9249ca9eb8b61814e78902" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40623 } } }
I20260810 06:49:56.923101 25563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:56.923154 25562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902 [sys.catalog]: This master's current role is: LEADER
I20260810 06:49:56.925381 25568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:49:56.930554 25568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:49:56.939119 25568 catalog_manager.cc:1347] Generated new cluster ID: 38d370b37ad54f4482b533d9cc837283
I20260810 06:49:56.939224 25568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:49:56.951867 25568 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:49:56.953126 25568 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:49:56.966586 25568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Generated new TSK 0
I20260810 06:49:56.967305 25568 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:49:57.022066 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40623
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:57.378677 25593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:57.378935 25593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:57.379045 25593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:57.385936 25593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:57.386034 25593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:57.386121 25593 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:49:57.386173 25593 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:49:57.386236 25593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:49:57.395359 25593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:40623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:57.396497 25593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:57.397737 25593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:57.409976 25604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:57.409989 25609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:57.410019 25613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:57.410825 25593 server_base.cc:1034] running on GCE node
I20260810 06:49:57.411482 25593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:57.412456 25593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:57.413671 25593 hybrid_clock.cc:648] HybridClock initialized: now 1786344597413612 us; error 51 us; skew 500 ppm
I20260810 06:49:57.413936 25593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:57 dist-test-slave-f7th krb5kdc[25385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344597, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:57.417007 25593 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:49:57.418408 25593 webserver.cc:460] Webserver started at http://127.22.11.129:41823/ using document root <none> and password file <none>
I20260810 06:49:57.419019 25593 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:57.419118 25593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:57.419374 25593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:57.422256 25593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "f01b9ac4b73441a58d8267d21edc3c26"
format_stamp: "Formatted at 2026-08-10 06:49:57 on dist-test-slave-f7th"
server_key: "f713424676acba97fd56cc02f0d961c2"
server_key_iv: "bed8850836ca28d221cb13b62892620f"
server_key_version: "encryptionkey@0"
I20260810 06:49:57.422933 25593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "f01b9ac4b73441a58d8267d21edc3c26"
format_stamp: "Formatted at 2026-08-10 06:49:57 on dist-test-slave-f7th"
server_key: "f713424676acba97fd56cc02f0d961c2"
server_key_iv: "bed8850836ca28d221cb13b62892620f"
server_key_version: "encryptionkey@0"
I20260810 06:49:57.427831 25593 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:49:57.430936 25627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:57.432093 25593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 06:49:57.432224 25593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "f01b9ac4b73441a58d8267d21edc3c26"
format_stamp: "Formatted at 2026-08-10 06:49:57 on dist-test-slave-f7th"
server_key: "f713424676acba97fd56cc02f0d961c2"
server_key_iv: "bed8850836ca28d221cb13b62892620f"
server_key_version: "encryptionkey@0"
I20260810 06:49:57.432384 25593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:57.449700 25593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:57.454095 25593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:57.454392 25593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:57.456955 25593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:57.457056 25593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:57.457170 25593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:57.457247 25593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:49:57 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344597, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:57.479780 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.457484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:43645 (local address 127.22.11.190:40623)
0810 06:49:57.459867 (+ 2383us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:57.459872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:57.459891 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:49:57.462148 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:57.462156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:57.462247 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:57.462355 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:57.464657 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.465477 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.467978 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.468423 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.468568 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.473111 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:57.473116 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:57.473119 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:57.473151 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:57.476103 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.476965 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.476972 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.476976 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.477129 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.477677 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.477683 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.477686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.478038 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:57.478172 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:57.479109 (+ 937us) server_negotiation.cc:299] Negotiation successful
0810 06:49:57.479359 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2265,"thread_start_us":2153,"threads_started":1}
I20260810 06:49:57.480605 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.458124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40623 (local address 127.22.11.129:43645)
0810 06:49:57.459103 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:57.459154 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:57.461734 (+ 2580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:57.462636 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:57.462651 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:57.463467 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:57.464423 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.464457 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.465677 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.465683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.467835 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.467852 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.468687 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.468695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.468980 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.469762 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:57.469790 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:57.472902 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:57.476234 (+ 3332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.476241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.476251 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.476840 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.477239 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.477244 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.477251 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.477537 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.478171 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:57.478183 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:57.478660 (+ 477us) client_negotiation.cc:769] Sending connection context
0810 06:49:57.478964 (+ 304us) client_negotiation.cc:240] Negotiation successful
0810 06:49:57.479372 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":393,"threads_started":1}
I20260810 06:49:57.499151 25593 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:46413
I20260810 06:49:57.499202 25839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:46413 every 8 connection(s)
I20260810 06:49:57.500346 25593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:49:57.501151 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25593
I20260810 06:49:57.501258 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.487810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40623 (local address 127.22.11.129:49619)
0810 06:49:57.488868 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:57.488890 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:57.489028 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:57.489341 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:57.489353 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:57.489971 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:57.490282 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.490294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.491272 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.491278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.492846 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.492866 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.493845 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.493849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.493965 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.494520 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:57.494545 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:57.495542 (+ 997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:57.498739 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.498745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.498749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.499182 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.499592 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.499597 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.499601 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.499725 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.500202 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:57.500206 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:57.500385 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 06:49:57.500508 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:49:57.500818 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":727,"threads_started":1}
I20260810 06:49:57.501379 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:49:57.501397 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.488239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:49619 (local address 127.22.11.190:40623)
0810 06:49:57.488398 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:57.488404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:57.488976 (+ 572us) server_negotiation.cc:407] Connection header received
0810 06:49:57.489143 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:57.489147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:57.489205 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:57.489283 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:57.490424 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.491106 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.492999 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.493678 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.493811 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.495730 (+ 1919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:57.495738 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:57.495743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:57.495780 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:57.498610 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.499334 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.499339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.499342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.499477 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.499822 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.499827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.499831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.500090 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:57.500218 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:57.500656 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 06:49:57.500906 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:49:57.502072 22574 external_mini_cluster.cc:1428] Setting key dd39686c5c8690bdd77ce628daf34be8
I20260810 06:49:57.510133 25840 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40623
I20260810 06:49:57.510367 25840 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:57.511085 25840 heartbeater.cc:511] Master 127.22.11.190:40623 requested a full tablet report, sending...
I20260810 06:49:57.513110 25463 ts_manager.cc:194] Registered new tserver with Master: f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413)
I20260810 06:49:57.514473 25463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:43645
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:49:57.545641 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40623
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:57.864672 25846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:57.864951 25846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:57.865089 25846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:57.872345 25846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:57.872447 25846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:57.872576 25846 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:49:57.872659 25846 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:49:57.872764 25846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:49:57.882368 25846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:40623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:57.883569 25846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:57.884833 25846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:57.897063 25852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:57.897136 25853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:57.897265 25855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:57.898252 25846 server_base.cc:1034] running on GCE node
I20260810 06:49:57.898818 25846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:57.899660 25846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:57.900852 25846 hybrid_clock.cc:648] HybridClock initialized: now 1786344597900806 us; error 33 us; skew 500 ppm
I20260810 06:49:57.901134 25846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:57 dist-test-slave-f7th krb5kdc[25385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344597, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:57.904053 25846 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:49:57.905390 25846 webserver.cc:460] Webserver started at http://127.22.11.130:36497/ using document root <none> and password file <none>
I20260810 06:49:57.905993 25846 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:57.906095 25846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:57.906348 25846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:57.909135 25846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "cd5a4dc3c93b4e09a375027c6e7854d9"
format_stamp: "Formatted at 2026-08-10 06:49:57 on dist-test-slave-f7th"
server_key: "d3bb0f4917fe07307f89790f366687f9"
server_key_iv: "b9212cf135592deced8ed36cd3dd5c75"
server_key_version: "encryptionkey@0"
I20260810 06:49:57.909819 25846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "cd5a4dc3c93b4e09a375027c6e7854d9"
format_stamp: "Formatted at 2026-08-10 06:49:57 on dist-test-slave-f7th"
server_key: "d3bb0f4917fe07307f89790f366687f9"
server_key_iv: "b9212cf135592deced8ed36cd3dd5c75"
server_key_version: "encryptionkey@0"
I20260810 06:49:57.914590 25846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:49:57.917784 25867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:57.918994 25846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:57.919137 25846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "cd5a4dc3c93b4e09a375027c6e7854d9"
format_stamp: "Formatted at 2026-08-10 06:49:57 on dist-test-slave-f7th"
server_key: "d3bb0f4917fe07307f89790f366687f9"
server_key_iv: "b9212cf135592deced8ed36cd3dd5c75"
server_key_version: "encryptionkey@0"
I20260810 06:49:57.919303 25846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:57.930308 25846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:57.933911 25846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:57.934159 25846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:57.936738 25846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:57.936854 25846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:57.936975 25846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:57.937081 25846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:49:57 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344597, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:57.959060 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.937308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:45445 (local address 127.22.11.190:40623)
0810 06:49:57.937479 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:57.937486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:57.938757 (+ 1271us) server_negotiation.cc:407] Connection header received
0810 06:49:57.940762 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:57.940768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:57.940850 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:57.940940 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:57.942917 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.943980 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.946477 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.947033 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.947158 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.951843 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:57.951850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:57.951854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:57.951886 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:57.955235 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.956159 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.956165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.956168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.956335 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.956923 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.956929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.956932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.957288 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:57.957437 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:57.958475 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0810 06:49:57.958714 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:49:57.960326 25883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.937867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40623 (local address 127.22.11.130:45445)
0810 06:49:57.938605 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:57.938642 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:57.940435 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:57.941194 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:57.941207 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:57.941878 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:57.942723 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.942754 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.944166 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.944172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.946333 (+ 2161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.946348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.947290 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.947299 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.947535 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.948379 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:57.948412 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:57.951559 (+ 3147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:57.955368 (+ 3809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.955377 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.955391 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.955990 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.956464 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.956469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.956472 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.956758 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.957519 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:57.957531 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:57.958004 (+ 473us) client_negotiation.cc:769] Sending connection context
0810 06:49:57.958352 (+ 348us) client_negotiation.cc:240] Negotiation successful
0810 06:49:57.958887 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":331,"threads_started":1}
I20260810 06:49:57.980520 25846 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:41225
I20260810 06:49:57.980767 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.966727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:60477 (local address 127.22.11.190:40623)
0810 06:49:57.966926 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:57.966932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:57.969150 (+ 2218us) server_negotiation.cc:407] Connection header received
0810 06:49:57.969328 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:57.969335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:57.969418 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:57.969532 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:57.970621 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.971306 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.972583 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.973076 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.973196 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.975319 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:57.975324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:57.975327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:57.975355 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:57.977875 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.978729 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.978734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.978737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.978878 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:57.979230 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:57.979233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:57.979234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:57.979637 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:57.979776 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:57.980237 (+ 461us) server_negotiation.cc:299] Negotiation successful
0810 06:49:57.980426 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:49:57.981048 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:57.967062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40623 (local address 127.22.11.130:60477)
0810 06:49:57.969031 (+ 1969us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:57.969051 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:57.969197 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:57.969547 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:57.969550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:57.969996 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:57.970477 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.970490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.971432 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.971438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.972453 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:57.972465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:57.973297 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:57.973304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:57.973513 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:57.974137 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:57.974163 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:57.975147 (+ 984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:57.977992 (+ 2845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.977998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.978002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.978602 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.978983 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:57.978987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:57.978991 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:57.979138 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:57.979834 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:57.979838 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:57.979994 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 06:49:57.980134 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 06:49:57.980517 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1805,"thread_start_us":1639,"threads_started":1}
I20260810 06:49:57.980748 26067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:41225 every 8 connection(s)
I20260810 06:49:57.982378 25846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:49:57.984725 26068 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40623
I20260810 06:49:57.984946 26068 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:57.985525 26068 heartbeater.cc:511] Master 127.22.11.190:40623 requested a full tablet report, sending...
I20260810 06:49:57.986763 25463 ts_manager.cc:194] Registered new tserver with Master: cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225)
I20260810 06:49:57.987546 25463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:45445
I20260810 06:49:57.992522 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25846
I20260810 06:49:57.992697 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:49:57.993131 22574 external_mini_cluster.cc:1428] Setting key f99125633dd42d1a55a353251c4cadd3
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:49:58.035063 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40623
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:49:58.349133 26076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:49:58.349390 26076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:49:58.349498 26076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:49:58.356323 26076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:49:58.356423 26076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:49:58.356511 26076 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 06:49:58.356570 26076 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:49:58.356664 26076 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:49:58.366158 26076 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:40623
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:49:58.367443 26076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:49:58.368702 26076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:49:58.380885 26088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:58.380929 26085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:49:58.380954 26084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:49:58.381968 26076 server_base.cc:1034] running on GCE node
I20260810 06:49:58.382575 26076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:49:58.383409 26076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:49:58.384598 26076 hybrid_clock.cc:648] HybridClock initialized: now 1786344598384551 us; error 37 us; skew 500 ppm
I20260810 06:49:58.384923 26076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:49:58 dist-test-slave-f7th krb5kdc[25385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344598, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:49:58.387807 26076 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:49:58.389194 26076 webserver.cc:460] Webserver started at http://127.22.11.131:42519/ using document root <none> and password file <none>
I20260810 06:49:58.389847 26076 fs_manager.cc:359] Metadata directory not provided
I20260810 06:49:58.389961 26076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:49:58.390249 26076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:49:58.393105 26076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "4347f2a8842642508eeb34f496ba404a"
format_stamp: "Formatted at 2026-08-10 06:49:58 on dist-test-slave-f7th"
server_key: "74c659546fb1a97fc29da15a38222486"
server_key_iv: "bc01652a6fb45874516973434ca9c52b"
server_key_version: "encryptionkey@0"
I20260810 06:49:58.393821 26076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "4347f2a8842642508eeb34f496ba404a"
format_stamp: "Formatted at 2026-08-10 06:49:58 on dist-test-slave-f7th"
server_key: "74c659546fb1a97fc29da15a38222486"
server_key_iv: "bc01652a6fb45874516973434ca9c52b"
server_key_version: "encryptionkey@0"
I20260810 06:49:58.398639 26076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:49:58.401867 26098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:58.403106 26076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:49:58.403251 26076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "4347f2a8842642508eeb34f496ba404a"
format_stamp: "Formatted at 2026-08-10 06:49:58 on dist-test-slave-f7th"
server_key: "74c659546fb1a97fc29da15a38222486"
server_key_iv: "bc01652a6fb45874516973434ca9c52b"
server_key_version: "encryptionkey@0"
I20260810 06:49:58.403429 26076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:49:58.416033 26076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:49:58.419602 26076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:49:58.419840 26076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:49:58.422555 26076 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:49:58.422688 26076 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:49:58.422796 26076 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:49:58.422875 26076 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:49:58 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344598, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:58.446077 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.422949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:50563 (local address 127.22.11.190:40623)
0810 06:49:58.423148 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.423154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.424463 (+ 1309us) server_negotiation.cc:407] Connection header received
0810 06:49:58.426800 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.426809 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.426914 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.427022 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:58.428574 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.429746 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.432198 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.432788 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.432912 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.438125 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:58.438133 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:58.438138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:58.438184 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:58.441657 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.442716 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.442722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.442726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.442907 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.443580 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.443587 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.443590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.443999 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:58.444153 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.445457 (+ 1304us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.445715 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:49:58.447818 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.423480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40623 (local address 127.22.11.131:50563)
0810 06:49:58.424289 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.424332 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.426516 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.427275 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.427286 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.427799 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:58.428413 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.428430 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.429932 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.429937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.432001 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.432017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.433221 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.433229 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.433506 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.434164 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:58.434204 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:58.437869 (+ 3665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:58.441813 (+ 3944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:58.441824 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:58.441847 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:58.442581 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:58.443053 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:58.443064 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:58.443068 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:58.443398 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:58.444257 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:58.444269 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:58.444881 (+ 612us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.445310 (+ 429us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.445956 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":339,"threads_started":1}
I20260810 06:49:58.463538 26076 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:39083
I20260810 06:49:58.463570 26297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:39083 every 8 connection(s)
I20260810 06:49:58.465193 26076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:49:58.467773 26298 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40623
I20260810 06:49:58.467973 26298 heartbeater.cc:464] Registering TS with master...
I20260810 06:49:58.468647 26298 heartbeater.cc:511] Master 127.22.11.190:40623 requested a full tablet report, sending...
I20260810 06:49:58.470170 25463 ts_manager.cc:194] Registered new tserver with Master: 4347f2a8842642508eeb34f496ba404a (127.22.11.131:39083)
I20260810 06:49:58.471133 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26076
I20260810 06:49:58.471154 25463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:50563
I20260810 06:49:58.471297 26268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.455959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40623 (local address 127.22.11.131:45745)
0810 06:49:58.457170 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.457185 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.457315 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.457802 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.457807 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.458444 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:58.458941 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.458960 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.460374 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.460380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.462145 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.462164 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.463242 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.463250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.463434 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.464203 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:58.464226 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:58.465260 (+ 1034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:58.468574 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:58.468579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:58.468582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:58.469161 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:58.469593 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:58.469596 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:58.469599 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:58.469714 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:58.470273 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:58.470278 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:58.470440 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.470566 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.470867 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"thread_start_us":868,"threads_started":1}
I20260810 06:49:58.471316 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:49:58.471318 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.455715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:45745 (local address 127.22.11.190:40623)
0810 06:49:58.456333 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.456339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.457267 (+ 928us) server_negotiation.cc:407] Connection header received
0810 06:49:58.457430 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.457436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.457528 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.457621 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:58.459126 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.460223 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.462337 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.463017 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.463162 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.465463 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:58.465469 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:58.465472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:58.465505 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:58.468439 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.469312 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.469316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.469319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.469464 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.469836 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.469840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.469842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.470152 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:58.470311 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.470694 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.470910 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459}
I20260810 06:49:58.471854 22574 external_mini_cluster.cc:1428] Setting key 5eec737e459b8355e8b78b7012080eac
I20260810 06:49:58.475068 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:49:58 dist-test-slave-f7th krb5kdc[25385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260810 06:49:58.516952 25840 heartbeater.cc:503] Master 127.22.11.190:40623 was elected leader, sending a full tablet report...
Aug 10 06:49:58 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:58.525772 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.512186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43646 (local address 127.22.11.190:40623)
0810 06:49:58.512420 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.512427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.512606 (+ 179us) server_negotiation.cc:407] Connection header received
0810 06:49:58.512827 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.512832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.512898 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.512990 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:58.514358 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.515101 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.516322 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.516800 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.516925 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.520986 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:58.521002 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:58.521005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:58.521037 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:58.523577 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.524141 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.524145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.524148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.524277 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.524546 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.524549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.524551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.524863 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:58.525068 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.525288 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.525480 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:49:58.529721 25463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:58.536520 25463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:58.558881 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.547878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:41225 (local address 127.0.0.1:50138)
0810 06:49:58.548647 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.548683 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.548848 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.551591 (+ 2743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.551599 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.551628 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:58.552090 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.552104 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.554003 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.554009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.556356 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.556374 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.557789 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.557796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.558112 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.558176 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.558287 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.558457 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":92,"thread_start_us":323,"threads_started":1}
I20260810 06:49:58.559705 26323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.548225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39083 (local address 127.0.0.1:50370)
0810 06:49:58.551494 (+ 3269us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.551513 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.551741 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.552715 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.552720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.552747 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:58.553140 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.553154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.554806 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.554811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.556835 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.556860 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.558507 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.558512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.558700 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.558736 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.558875 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.558994 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3137,"mutex_wait_us":1,"thread_start_us":1988,"threads_started":1}
I20260810 06:49:58.559701 26324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.548921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.22.11.131:39083)
0810 06:49:58.550771 (+ 1850us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.550780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.551617 (+ 837us) server_negotiation.cc:407] Connection header received
0810 06:49:58.551947 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.551956 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.552193 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.552350 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:58.553336 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.554688 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.557005 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.558317 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.558447 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.558679 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.559202 (+ 523us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.559378 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1676,"thread_start_us":1624,"threads_started":1}
I20260810 06:49:58.560294 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.548124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50138 (local address 127.22.11.130:41225)
0810 06:49:58.550772 (+ 2648us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.550780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.550808 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:49:58.551020 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.551033 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.551258 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.552266 (+ 1008us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:58.552457 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.553865 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.556522 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.557584 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.557709 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.557917 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.559312 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.560080 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2516,"thread_start_us":2388,"threads_started":1}
I20260810 06:49:58.561096 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.550194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:46413 (local address 127.0.0.1:36360)
0810 06:49:58.552651 (+ 2457us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.552668 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.552785 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.553729 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.553734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.553760 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:58.554239 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.554254 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.556022 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.556027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.558161 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.558176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.560319 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.560325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.560484 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.560527 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.560639 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.560757 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2315,"mutex_wait_us":73,"spinlock_wait_cycles":4480,"thread_start_us":871,"threads_started":1}
I20260810 06:49:58.561136 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.550413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.22.11.129:46413)
0810 06:49:58.553224 (+ 2811us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.553232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.553258 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:49:58.553353 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.553360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.553546 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.553709 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:58.554434 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.555866 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.558319 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.559723 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.560006 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.560198 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.560710 (+ 512us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.560863 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2668,"thread_start_us":2520,"threads_started":1}
I20260810 06:49:58.563413 26172 tablet_service.cc:1467] Processing CreateTablet for tablet 2b6002e7f237439892f36eb0ea816b26 (DEFAULT_TABLE table=test-table [id=8bf7935fd93a47a097317f78624fd75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:58.563707 25959 tablet_service.cc:1467] Processing CreateTablet for tablet 2b6002e7f237439892f36eb0ea816b26 (DEFAULT_TABLE table=test-table [id=8bf7935fd93a47a097317f78624fd75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:58.564829 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b6002e7f237439892f36eb0ea816b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:58.564541 25713 tablet_service.cc:1467] Processing CreateTablet for tablet 2b6002e7f237439892f36eb0ea816b26 (DEFAULT_TABLE table=test-table [id=8bf7935fd93a47a097317f78624fd75c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:58.565444 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b6002e7f237439892f36eb0ea816b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:58.565843 25713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b6002e7f237439892f36eb0ea816b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:58.575510 26331 tablet_bootstrap.cc:492] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Bootstrap starting.
I20260810 06:49:58.579332 26331 tablet_bootstrap.cc:654] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:58.580305 26331 log.cc:824] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:58.583065 26332 tablet_bootstrap.cc:492] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Bootstrap starting.
I20260810 06:49:58.585811 26334 tablet_bootstrap.cc:492] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Bootstrap starting.
I20260810 06:49:58.587098 26332 tablet_bootstrap.cc:654] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:58.588459 26332 log.cc:824] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:58.589443 26334 tablet_bootstrap.cc:654] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:58.590538 26334 log.cc:824] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:49:58.591661 26331 tablet_bootstrap.cc:492] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: No bootstrap required, opened a new log
I20260810 06:49:58.591658 26332 tablet_bootstrap.cc:492] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: No bootstrap required, opened a new log
I20260810 06:49:58.591938 26332 ts_tablet_manager.cc:1397] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 06:49:58.591938 26331 ts_tablet_manager.cc:1397] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260810 06:49:58.593884 26334 tablet_bootstrap.cc:492] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: No bootstrap required, opened a new log
I20260810 06:49:58.594115 26334 ts_tablet_manager.cc:1397] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:49:58.600988 26334 raft_consensus.cc:348] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.601313 26334 raft_consensus.cc:374] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:58.601408 26334 raft_consensus.cc:729] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5a4dc3c93b4e09a375027c6e7854d9, State: Initialized, Role: FOLLOWER
I20260810 06:49:58.601938 26334 consensus_queue.cc:260] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.602232 26332 raft_consensus.cc:348] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.602232 26331 raft_consensus.cc:348] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.602644 26332 raft_consensus.cc:374] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:58.602644 26331 raft_consensus.cc:374] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:58.602746 26331 raft_consensus.cc:729] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Initialized, Role: FOLLOWER
I20260810 06:49:58.602751 26332 raft_consensus.cc:729] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01b9ac4b73441a58d8267d21edc3c26, State: Initialized, Role: FOLLOWER
I20260810 06:49:58.603065 26068 heartbeater.cc:503] Master 127.22.11.190:40623 was elected leader, sending a full tablet report...
I20260810 06:49:58.603317 26331 consensus_queue.cc:260] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.603552 26334 ts_tablet_manager.cc:1428] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent starting tablet: real 0.009s user 0.006s sys 0.004s
I20260810 06:49:58.603507 26332 consensus_queue.cc:260] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.605055 26298 heartbeater.cc:503] Master 127.22.11.190:40623 was elected leader, sending a full tablet report...
I20260810 06:49:58.605976 26331 ts_tablet_manager.cc:1428] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260810 06:49:58.606657 26332 ts_tablet_manager.cc:1428] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260810 06:49:58.644899 26342 raft_consensus.cc:482] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:58.645279 26342 raft_consensus.cc:504] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.646991 26342 leader_election.cc:288] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225), f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413)
I20260810 06:49:58.653092 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.647420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:58231 (local address 127.22.11.130:41225)
0810 06:49:58.647618 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.647623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.647642 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:49:58.647794 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.647799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.647876 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.647967 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:58.648390 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.649349 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.651235 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.652282 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.652404 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.652456 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.652798 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.652892 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:49:58.653170 26113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.647161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:41225 (local address 127.22.11.131:58231)
0810 06:49:58.647476 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.647495 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.647615 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.648018 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.648023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.648053 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:58.648228 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.648242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.649468 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.649474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.651040 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.651052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.652420 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.652426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.652646 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.652683 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.652780 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.652911 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260810 06:49:58.654066 26326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.647521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:36289 (local address 127.22.11.129:46413)
0810 06:49:58.647740 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.647746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.648281 (+ 535us) server_negotiation.cc:407] Connection header received
0810 06:49:58.648395 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.648399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.648457 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.648525 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:49:58.649121 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.650062 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.652052 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.653141 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.653293 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.653363 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.653709 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.653837 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:49:58.654131 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.647383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:46413 (local address 127.22.11.131:36289)
0810 06:49:58.648176 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.648189 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.648289 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.648557 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.648561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.648586 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:49:58.648978 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.648990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.650194 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.650200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.651900 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.651915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.653428 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.653433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.653557 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.653579 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.653658 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.653767 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":1,"thread_start_us":443,"threads_started":1}
I20260810 06:49:58.654114 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6002e7f237439892f36eb0ea816b26" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" is_pre_election: true
I20260810 06:49:58.654668 25989 raft_consensus.cc:2454] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:49:58.655160 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6002e7f237439892f36eb0ea816b26" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26" is_pre_election: true
I20260810 06:49:58.655392 26106 leader_election.cc:302] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, cd5a4dc3c93b4e09a375027c6e7854d9; no voters:
I20260810 06:49:58.655692 25745 raft_consensus.cc:2454] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:49:58.655791 26342 raft_consensus.cc:2781] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:58.655923 26342 raft_consensus.cc:482] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:58.656008 26342 raft_consensus.cc:3037] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:58.658303 26342 raft_consensus.cc:504] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.658859 26342 leader_election.cc:288] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Requested vote from peers cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225), f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413)
I20260810 06:49:58.659330 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6002e7f237439892f36eb0ea816b26" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9"
I20260810 06:49:58.659332 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b6002e7f237439892f36eb0ea816b26" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26"
I20260810 06:49:58.659538 25745 raft_consensus.cc:3037] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:58.659560 25989 raft_consensus.cc:3037] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:58.662791 25745 raft_consensus.cc:2454] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:49:58.662791 25989 raft_consensus.cc:2454] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:49:58.663177 26106 leader_election.cc:302] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, cd5a4dc3c93b4e09a375027c6e7854d9; no voters:
I20260810 06:49:58.663424 26342 raft_consensus.cc:2781] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:58.663872 26342 raft_consensus.cc:686] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 1 LEADER]: Becoming Leader. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Running, Role: LEADER
I20260810 06:49:58.664305 26342 consensus_queue.cc:237] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } }
I20260810 06:49:58.669225 25464 catalog_manager.cc:5495] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4347f2a8842642508eeb34f496ba404a (127.22.11.131). New cstate: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } health_report { overall_health: UNKNOWN } } }
W20260810 06:49:58.716885 26300 tablet.cc:2369] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:49:58 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344596, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:49:58.732123 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.719096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40623 (local address 127.0.0.1:43662)
0810 06:49:58.719651 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:58.719666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:58.719792 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:58.720103 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:58.720106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:58.720921 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:49:58.721189 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.721199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.722278 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.722283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.723392 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:58.723406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.724112 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.724117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:58.724253 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.724858 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:49:58.724876 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:49:58.727113 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:49:58.729732 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:58.729742 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:58.729748 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:58.730244 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:58.730624 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:49:58.730629 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:49:58.730633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:49:58.730780 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:49:58.731280 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:49:58.731287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:49:58.731428 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 06:49:58.731568 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 06:49:58.731765 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":294,"threads_started":1}
I20260810 06:49:58.732165 25643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:58.719056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.22.11.190:40623)
0810 06:49:58.719208 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:58.719212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:58.719751 (+ 539us) server_negotiation.cc:407] Connection header received
0810 06:49:58.719881 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:58.719885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:58.719961 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:58.720051 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:49:58.721333 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.722141 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.723515 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:58.723952 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:58.724049 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:58.727272 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:49:58.727278 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:49:58.727281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:49:58.727309 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:49:58.729596 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.730350 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.730355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.730357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.730515 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:49:58.730872 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:49:58.730876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:49:58.730877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:49:58.731160 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:49:58.731269 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:58.731660 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 06:49:58.731883 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":48}
W20260810 06:49:58.734479 26071 tablet.cc:2369] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:58.737054 25464 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:43662:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:49:58.738005 25464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:58.744149 25959 tablet_service.cc:1467] Processing CreateTablet for tablet b5610c486e1548a6803388d0178f425d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aba372f172db4110aa56f2e5cc6f3ac0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:49:58.744140 25713 tablet_service.cc:1467] Processing CreateTablet for tablet b5610c486e1548a6803388d0178f425d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aba372f172db4110aa56f2e5cc6f3ac0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:49:58.744309 26172 tablet_service.cc:1467] Processing CreateTablet for tablet b5610c486e1548a6803388d0178f425d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aba372f172db4110aa56f2e5cc6f3ac0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:49:58.745157 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5610c486e1548a6803388d0178f425d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:58.745167 25959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5610c486e1548a6803388d0178f425d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:58.745167 25713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5610c486e1548a6803388d0178f425d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:58.750645 26331 tablet_bootstrap.cc:492] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: Bootstrap starting.
I20260810 06:49:58.752123 26331 tablet_bootstrap.cc:654] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:58.752158 26332 tablet_bootstrap.cc:492] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: Bootstrap starting.
I20260810 06:49:58.752522 26334 tablet_bootstrap.cc:492] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: Bootstrap starting.
I20260810 06:49:58.753818 26332 tablet_bootstrap.cc:654] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:58.754015 26334 tablet_bootstrap.cc:654] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:58.756129 26334 tablet_bootstrap.cc:492] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: No bootstrap required, opened a new log
I20260810 06:49:58.756263 26334 ts_tablet_manager.cc:1397] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260810 06:49:58.756317 26332 tablet_bootstrap.cc:492] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: No bootstrap required, opened a new log
I20260810 06:49:58.756474 26332 ts_tablet_manager.cc:1397] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260810 06:49:58.757175 26331 tablet_bootstrap.cc:492] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: No bootstrap required, opened a new log
I20260810 06:49:58.757071 26334 raft_consensus.cc:348] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:58.757323 26331 ts_tablet_manager.cc:1397] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260810 06:49:58.757328 26334 raft_consensus.cc:374] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:58.757413 26334 raft_consensus.cc:729] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5a4dc3c93b4e09a375027c6e7854d9, State: Initialized, Role: FOLLOWER
I20260810 06:49:58.757277 26332 raft_consensus.cc:348] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:58.757510 26332 raft_consensus.cc:374] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:58.757593 26332 raft_consensus.cc:729] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01b9ac4b73441a58d8267d21edc3c26, State: Initialized, Role: FOLLOWER
I20260810 06:49:58.757644 26334 consensus_queue.cc:260] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:58.757817 26332 consensus_queue.cc:260] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:58.758116 26331 raft_consensus.cc:348] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:58.758267 26334 ts_tablet_manager.cc:1428] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:49:58.758335 26331 raft_consensus.cc:374] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:58.758417 26331 raft_consensus.cc:729] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Initialized, Role: FOLLOWER
I20260810 06:49:58.758142 26341 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:58.758248 26343 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:58.758534 26341 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:58.758569 26343 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:58.758627 26331 consensus_queue.cc:260] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:58.759150 26331 ts_tablet_manager.cc:1428] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:49:58.760032 26332 ts_tablet_manager.cc:1428] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
W20260810 06:49:58.760457 25841 tablet.cc:2369] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:49:58.758984 26342 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:58.760797 26342 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:59.127743 26342 raft_consensus.cc:482] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:59.127944 26342 raft_consensus.cc:504] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.128513 26342 leader_election.cc:288] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225)
I20260810 06:49:59.129086 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5610c486e1548a6803388d0178f425d" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" is_pre_election: true
I20260810 06:49:59.129087 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5610c486e1548a6803388d0178f425d" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26" is_pre_election: true
I20260810 06:49:59.129356 25745 raft_consensus.cc:2454] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:49:59.129359 25989 raft_consensus.cc:2454] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:49:59.129765 26100 leader_election.cc:302] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, f01b9ac4b73441a58d8267d21edc3c26; no voters:
I20260810 06:49:59.130048 26342 raft_consensus.cc:2781] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:59.130156 26342 raft_consensus.cc:482] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:59.130257 26342 raft_consensus.cc:3037] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:59.132277 26342 raft_consensus.cc:504] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.132779 26342 leader_election.cc:288] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Requested vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225)
I20260810 06:49:59.133142 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5610c486e1548a6803388d0178f425d" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26"
I20260810 06:49:59.133232 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5610c486e1548a6803388d0178f425d" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9"
I20260810 06:49:59.133337 25745 raft_consensus.cc:3037] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:59.133423 25989 raft_consensus.cc:3037] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:59.135331 25989 raft_consensus.cc:2454] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:49:59.135370 25745 raft_consensus.cc:2454] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:49:59.135707 26106 leader_election.cc:302] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, cd5a4dc3c93b4e09a375027c6e7854d9; no voters:
I20260810 06:49:59.136000 26342 raft_consensus.cc:2781] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:59.136152 26342 raft_consensus.cc:686] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [term 1 LEADER]: Becoming Leader. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Running, Role: LEADER
I20260810 06:49:59.136405 26342 consensus_queue.cc:237] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.137933 26348 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: TxnStatusTablet state changed. Reason: New leader 4347f2a8842642508eeb34f496ba404a. Latest consensus state: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:59.138182 26348 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:49:59.138688 26364 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:49:59.138957 26364 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:49:59.139303 25464 catalog_manager.cc:5495] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4347f2a8842642508eeb34f496ba404a (127.22.11.131). New cstate: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } health_report { overall_health: HEALTHY } } }
I20260810 06:49:59.184774 26348 consensus_queue.cc:1035] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:59.192472 26348 consensus_queue.cc:1035] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:59.218014 26357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:59.211666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39083 (local address 127.0.0.1:50374)
0810 06:49:59.211921 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:59.211945 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:59.212097 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:59.212645 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:59.212650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:59.212662 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:49:59.212823 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:59.212830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.214283 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.214287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:59.215246 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:59.215259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.216242 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.216248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:59.216393 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:59.216449 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:49:59.217464 (+ 1015us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:49:59.217499 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:49:59.217614 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 06:49:59.217765 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:49:59.218130 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:59.211666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.22.11.131:39083)
0810 06:49:59.212229 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:59.212236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:59.212254 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:49:59.212350 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:59.212355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:59.212454 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:59.212640 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:59.212967 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.214183 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.215379 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.216072 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.216198 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:59.216650 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:59.217323 (+ 673us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:59.217463 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:59.217667 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:49:59.217788 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":285,"threads_started":1}
I20260810 06:49:59.224468 25989 raft_consensus.cc:1261] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Refusing update from remote peer 4347f2a8842642508eeb34f496ba404a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:59.224824 25745 raft_consensus.cc:1261] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Refusing update from remote peer 4347f2a8842642508eeb34f496ba404a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:59.225134 26342 consensus_queue.cc:1035] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:59.225472 26348 consensus_queue.cc:1035] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:59.239933 26343 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: TxnStatusTablet state changed. Reason: New leader 4347f2a8842642508eeb34f496ba404a. Latest consensus state: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:59.240262 26343 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:59.243750 26348 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:59.243996 26348 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:49:59.244474 26372 mvcc.cc:205] Tried to move back new op lower bound from 7316867478417227776 to 7316867478065065984. Current Snapshot: MvccSnapshot[applied={T|T < 7316867478417227776}]
I20260810 06:49:59.247237 26342 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:59.247520 26342 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P 4347f2a8842642508eeb34f496ba404a: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:49:59.248499 26341 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: TxnStatusTablet state changed. Reason: New leader 4347f2a8842642508eeb34f496ba404a. Latest consensus state: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:59.248804 26341 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:59.258584 26343 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:59.258826 26343 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P f01b9ac4b73441a58d8267d21edc3c26: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:59.263123 26341 tablet_replica.cc:439] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } } }
I20260810 06:49:59.263358 26341 tablet_replica.cc:442] T b5610c486e1548a6803388d0178f425d P cd5a4dc3c93b4e09a375027c6e7854d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:49:59.273440 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:59.262581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.22.11.131:39083)
0810 06:49:59.262757 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:59.262781 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:59.266776 (+ 3995us) server_negotiation.cc:407] Connection header received
0810 06:49:59.266983 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:59.266988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:59.267069 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:59.267152 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:59.267799 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.268924 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.270885 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.271363 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.271475 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:59.271961 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:59.272091 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:59.272185 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:59.273050 (+ 865us) server_negotiation.cc:299] Negotiation successful
0810 06:49:59.273138 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:49:59.289801 26374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:59.283721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:45715 (local address 127.22.11.131:39083)
0810 06:49:59.283921 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:49:59.283927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:49:59.284498 (+ 571us) server_negotiation.cc:407] Connection header received
0810 06:49:59.284729 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:49:59.284732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:49:59.284797 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:49:59.284863 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:49:59.285353 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.286092 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.288170 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.288555 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.288669 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:59.289159 (+ 490us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:49:59.289247 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:49:59.289308 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:49:59.289442 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 06:49:59.289550 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":63}
I20260810 06:49:59.290094 26388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:49:59.283712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39083 (local address 127.22.11.131:45715)
0810 06:49:59.284392 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 06:49:59.284416 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:49:59.284572 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:49:59.284891 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:49:59.284896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:49:59.284908 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:49:59.285201 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:59.285212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.286229 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.286235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:59.288039 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:49:59.288056 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:49:59.288712 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:49:59.288718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:49:59.288904 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:49:59.288968 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:49:59.289338 (+ 370us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:49:59.289349 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:49:59.289433 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 06:49:59.289622 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":22,"thread_start_us":380,"threads_started":1}
I20260810 06:49:59.353199 25464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:59.353438 25464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:49:59.356478 25464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Sending DeleteTablet for 3 replicas of tablet 2b6002e7f237439892f36eb0ea816b26
I20260810 06:49:59.357462 25959 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b6002e7f237439892f36eb0ea816b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:59 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50138
I20260810 06:49:59.357743 25713 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b6002e7f237439892f36eb0ea816b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:59 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:36360
I20260810 06:49:59.358256 26172 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b6002e7f237439892f36eb0ea816b26 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:49:59 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 06:49:59.359030 25464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:49:59.359694 25464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:49:59.367390 25712 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec6fa2273e64ad588c03a93df2c5e24 (DEFAULT_TABLE table=test-table [id=e1ca06659b4c461fa72c966c855afb13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:59.367986 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec6fa2273e64ad588c03a93df2c5e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:59.372956 26396 tablet_replica.cc:330] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: stopping tablet replica
I20260810 06:49:59.373466 26171 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec6fa2273e64ad588c03a93df2c5e24 (DEFAULT_TABLE table=test-table [id=e1ca06659b4c461fa72c966c855afb13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:59.373865 25956 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec6fa2273e64ad588c03a93df2c5e24 (DEFAULT_TABLE table=test-table [id=e1ca06659b4c461fa72c966c855afb13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:49:59.374028 26171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec6fa2273e64ad588c03a93df2c5e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:59.374292 25956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec6fa2273e64ad588c03a93df2c5e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:49:59.383013 26396 raft_consensus.cc:2229] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:59.383697 26395 tablet_replica.cc:330] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: stopping tablet replica
I20260810 06:49:59.383793 26404 tablet_bootstrap.cc:492] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: Bootstrap starting.
I20260810 06:49:59.384105 26395 raft_consensus.cc:2229] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:49:59.384465 26395 raft_consensus.cc:2258] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:59.386015 26404 tablet_bootstrap.cc:654] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:59.387885 26396 raft_consensus.cc:2258] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:59.389031 26403 tablet_bootstrap.cc:492] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: Bootstrap starting.
I20260810 06:49:59.389729 26396 ts_tablet_manager.cc:1905] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:59.390656 26398 tablet_bootstrap.cc:492] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: Bootstrap starting.
I20260810 06:49:59.389324 26395 ts_tablet_manager.cc:1905] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:59.391599 26397 tablet_replica.cc:330] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: stopping tablet replica
I20260810 06:49:59.392211 26403 tablet_bootstrap.cc:654] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:59.392688 26397 raft_consensus.cc:2229] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:49:59.394445 26397 raft_consensus.cc:2258] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:49:59.396268 26397 ts_tablet_manager.cc:1905] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:49:59.397562 26398 tablet_bootstrap.cc:654] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: Neither blocks nor log segments found. Creating new log.
I20260810 06:49:59.398514 26403 tablet_bootstrap.cc:492] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: No bootstrap required, opened a new log
I20260810 06:49:59.398787 26403 ts_tablet_manager.cc:1397] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260810 06:49:59.399308 26395 ts_tablet_manager.cc:1918] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:49:59.399492 26395 log.cc:1196] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/2b6002e7f237439892f36eb0ea816b26
I20260810 06:49:59.399598 26404 tablet_bootstrap.cc:492] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: No bootstrap required, opened a new log
I20260810 06:49:59.399746 26404 ts_tablet_manager.cc:1397] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.004s
I20260810 06:49:59.400043 26395 ts_tablet_manager.cc:1939] T 2b6002e7f237439892f36eb0ea816b26 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting consensus metadata
I20260810 06:49:59.399972 26403 raft_consensus.cc:348] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.400344 26403 raft_consensus.cc:374] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:59.400480 26403 raft_consensus.cc:729] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5a4dc3c93b4e09a375027c6e7854d9, State: Initialized, Role: FOLLOWER
I20260810 06:49:59.400636 26404 raft_consensus.cc:348] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.400889 26404 raft_consensus.cc:374] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:59.401002 26404 raft_consensus.cc:729] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Initialized, Role: FOLLOWER
I20260810 06:49:59.400998 26403 consensus_queue.cc:260] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.401290 26404 consensus_queue.cc:260] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.403215 26404 ts_tablet_manager.cc:1428] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:49:59.403769 25439 catalog_manager.cc:4841] TS cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225): tablet 2b6002e7f237439892f36eb0ea816b26 (table test-table [id=8bf7935fd93a47a097317f78624fd75c]) successfully deleted
I20260810 06:49:59.404785 26403 ts_tablet_manager.cc:1428] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 06:49:59.405377 26396 ts_tablet_manager.cc:1918] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:49:59.405563 26396 log.cc:1196] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/2b6002e7f237439892f36eb0ea816b26
I20260810 06:49:59.406081 26396 ts_tablet_manager.cc:1939] T 2b6002e7f237439892f36eb0ea816b26 P f01b9ac4b73441a58d8267d21edc3c26: Deleting consensus metadata
I20260810 06:49:59.407168 26397 ts_tablet_manager.cc:1918] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:49:59.407380 26397 log.cc:1196] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/2b6002e7f237439892f36eb0ea816b26
I20260810 06:49:59.407889 26397 ts_tablet_manager.cc:1939] T 2b6002e7f237439892f36eb0ea816b26 P 4347f2a8842642508eeb34f496ba404a: Deleting consensus metadata
I20260810 06:49:59.408099 25437 catalog_manager.cc:4841] TS f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413): tablet 2b6002e7f237439892f36eb0ea816b26 (table test-table [id=8bf7935fd93a47a097317f78624fd75c]) successfully deleted
I20260810 06:49:59.409086 26398 tablet_bootstrap.cc:492] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: No bootstrap required, opened a new log
I20260810 06:49:59.409228 26398 ts_tablet_manager.cc:1397] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.000s
I20260810 06:49:59.409869 25440 catalog_manager.cc:4841] TS 4347f2a8842642508eeb34f496ba404a (127.22.11.131:39083): tablet 2b6002e7f237439892f36eb0ea816b26 (table test-table [id=8bf7935fd93a47a097317f78624fd75c]) successfully deleted
I20260810 06:49:59.409940 26398 raft_consensus.cc:348] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.410174 26398 raft_consensus.cc:374] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:49:59.410264 26398 raft_consensus.cc:729] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01b9ac4b73441a58d8267d21edc3c26, State: Initialized, Role: FOLLOWER
I20260810 06:49:59.410477 26398 consensus_queue.cc:260] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.410957 26398 ts_tablet_manager.cc:1428] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:49:59.505736 26391 raft_consensus.cc:482] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:49:59.505926 26391 raft_consensus.cc:504] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.506714 26391 leader_election.cc:288] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225)
I20260810 06:49:59.507232 25744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec6fa2273e64ad588c03a93df2c5e24" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26" is_pre_election: true
I20260810 06:49:59.507433 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec6fa2273e64ad588c03a93df2c5e24" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" is_pre_election: true
I20260810 06:49:59.507604 25744 raft_consensus.cc:2454] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:49:59.507678 25989 raft_consensus.cc:2454] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:49:59.508181 26106 leader_election.cc:302] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, cd5a4dc3c93b4e09a375027c6e7854d9; no voters:
I20260810 06:49:59.508519 26391 raft_consensus.cc:2781] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:49:59.508653 26391 raft_consensus.cc:482] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:49:59.508761 26391 raft_consensus.cc:3037] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:59.511114 26391 raft_consensus.cc:504] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.511698 26391 leader_election.cc:288] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Requested vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225)
I20260810 06:49:59.512233 25744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec6fa2273e64ad588c03a93df2c5e24" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26"
I20260810 06:49:59.512308 25989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec6fa2273e64ad588c03a93df2c5e24" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9"
I20260810 06:49:59.512460 25744 raft_consensus.cc:3037] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:59.512490 25989 raft_consensus.cc:3037] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:49:59.515334 25989 raft_consensus.cc:2454] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:49:59.515435 25744 raft_consensus.cc:2454] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:49:59.515864 26100 leader_election.cc:302] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, f01b9ac4b73441a58d8267d21edc3c26; no voters:
I20260810 06:49:59.516170 26391 raft_consensus.cc:2781] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:49:59.516366 26391 raft_consensus.cc:686] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 1 LEADER]: Becoming Leader. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Running, Role: LEADER
I20260810 06:49:59.516718 26391 consensus_queue.cc:237] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:49:59.519320 25464 catalog_manager.cc:5495] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4347f2a8842642508eeb34f496ba404a (127.22.11.131). New cstate: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } health_report { overall_health: HEALTHY } } }
I20260810 06:49:59.549165 25989 raft_consensus.cc:1261] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Refusing update from remote peer 4347f2a8842642508eeb34f496ba404a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:59.549254 25744 raft_consensus.cc:1261] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Refusing update from remote peer 4347f2a8842642508eeb34f496ba404a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:49:59.549772 26391 consensus_queue.cc:1035] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:59.550034 26348 consensus_queue.cc:1035] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:49:59.554435 26369 mvcc.cc:205] Tried to move back new op lower bound from 7316867479748210688 to 7316867479622791168. Current Snapshot: MvccSnapshot[applied={T|T < 7316867479748210688}]
I20260810 06:49:59.555428 26370 mvcc.cc:205] Tried to move back new op lower bound from 7316867479748210688 to 7316867479622791168. Current Snapshot: MvccSnapshot[applied={T|T < 7316867479748210688}]
W20260810 06:49:59.572971 26369 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:50:01.156004 25463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:01.156250 25463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:01.160045 25463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Sending DeleteTablet for 3 replicas of tablet 0ec6fa2273e64ad588c03a93df2c5e24
I20260810 06:50:01.160801 25712 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec6fa2273e64ad588c03a93df2c5e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:01 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:36360
I20260810 06:50:01.160869 25956 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec6fa2273e64ad588c03a93df2c5e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:01 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50138
I20260810 06:50:01.160979 26172 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec6fa2273e64ad588c03a93df2c5e24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:01 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 06:50:01.173492 26448 tablet_replica.cc:330] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: stopping tablet replica
I20260810 06:50:01.173738 26448 raft_consensus.cc:2229] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:01.174015 26448 raft_consensus.cc:2258] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:01.174556 26446 tablet_replica.cc:330] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: stopping tablet replica
I20260810 06:50:01.174777 26446 raft_consensus.cc:2229] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:01.175040 26446 raft_consensus.cc:2258] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:01.175339 26449 tablet_replica.cc:330] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: stopping tablet replica
I20260810 06:50:01.175590 26449 raft_consensus.cc:2229] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:01.175856 26448 ts_tablet_manager.cc:1905] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:01.176023 26449 raft_consensus.cc:2258] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:01.176079 26446 ts_tablet_manager.cc:1905] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:01.176990 26449 ts_tablet_manager.cc:1905] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:01.182572 26448 ts_tablet_manager.cc:1918] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:50:01.182571 26446 ts_tablet_manager.cc:1918] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:50:01.182715 26446 log.cc:1196] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/0ec6fa2273e64ad588c03a93df2c5e24
I20260810 06:50:01.182750 26448 log.cc:1196] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/0ec6fa2273e64ad588c03a93df2c5e24
I20260810 06:50:01.183230 26448 ts_tablet_manager.cc:1939] T 0ec6fa2273e64ad588c03a93df2c5e24 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting consensus metadata
I20260810 06:50:01.183252 26446 ts_tablet_manager.cc:1939] T 0ec6fa2273e64ad588c03a93df2c5e24 P f01b9ac4b73441a58d8267d21edc3c26: Deleting consensus metadata
I20260810 06:50:01.183763 26449 ts_tablet_manager.cc:1918] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:50:01.183941 26449 log.cc:1196] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/0ec6fa2273e64ad588c03a93df2c5e24
I20260810 06:50:01.184365 25439 catalog_manager.cc:4841] TS cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225): tablet 0ec6fa2273e64ad588c03a93df2c5e24 (table test-table [id=e1ca06659b4c461fa72c966c855afb13]) successfully deleted
I20260810 06:50:01.184437 26449 ts_tablet_manager.cc:1939] T 0ec6fa2273e64ad588c03a93df2c5e24 P 4347f2a8842642508eeb34f496ba404a: Deleting consensus metadata
I20260810 06:50:01.184715 25437 catalog_manager.cc:4841] TS f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413): tablet 0ec6fa2273e64ad588c03a93df2c5e24 (table test-table [id=e1ca06659b4c461fa72c966c855afb13]) successfully deleted
I20260810 06:50:01.185384 25440 catalog_manager.cc:4841] TS 4347f2a8842642508eeb34f496ba404a (127.22.11.131:39083): tablet 0ec6fa2273e64ad588c03a93df2c5e24 (table test-table [id=e1ca06659b4c461fa72c966c855afb13]) successfully deleted
Aug 10 06:50:01 dist-test-slave-f7th krb5kdc[25385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:50:01 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:01.222091 26472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.204408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45550 (local address 127.22.11.190:40623)
0810 06:50:01.204998 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:01.205005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:01.205032 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:50:01.205226 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:01.205230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:01.205292 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:01.205382 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:01.206681 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.207524 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.208777 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.209186 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.209301 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.217405 (+ 8104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:01.217410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:01.217413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:01.217447 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:01.219867 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:01.220452 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:01.220456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:01.220458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:01.220641 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:01.220945 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:01.220948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:01.220950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:01.221198 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:01.221368 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:01.221640 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 06:50:01.221813 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":333,"threads_started":1}
I20260810 06:50:01.225238 25463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:01.225834 25463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:01.231895 25712 tablet_service.cc:1467] Processing CreateTablet for tablet e059c57c9ad840cc9e6af75e4566ca05 (DEFAULT_TABLE table=test-table [id=a4a8121e8d2047099c6c61c59a5a77b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:01.231896 25956 tablet_service.cc:1467] Processing CreateTablet for tablet e059c57c9ad840cc9e6af75e4566ca05 (DEFAULT_TABLE table=test-table [id=a4a8121e8d2047099c6c61c59a5a77b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:01.232141 26172 tablet_service.cc:1467] Processing CreateTablet for tablet e059c57c9ad840cc9e6af75e4566ca05 (DEFAULT_TABLE table=test-table [id=a4a8121e8d2047099c6c61c59a5a77b2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:01.232375 25956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e059c57c9ad840cc9e6af75e4566ca05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:01.232379 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e059c57c9ad840cc9e6af75e4566ca05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:01.232515 26172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e059c57c9ad840cc9e6af75e4566ca05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:01.239360 26476 tablet_bootstrap.cc:492] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: Bootstrap starting.
I20260810 06:50:01.240031 26479 tablet_bootstrap.cc:492] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: Bootstrap starting.
I20260810 06:50:01.241043 26473 tablet_bootstrap.cc:492] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: Bootstrap starting.
I20260810 06:50:01.242012 26476 tablet_bootstrap.cc:654] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:01.242336 26473 tablet_bootstrap.cc:654] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:01.243837 26479 tablet_bootstrap.cc:654] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:01.244637 26473 tablet_bootstrap.cc:492] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: No bootstrap required, opened a new log
I20260810 06:50:01.244779 26473 ts_tablet_manager.cc:1397] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260810 06:50:01.245569 26473 raft_consensus.cc:348] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.245748 26473 raft_consensus.cc:374] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:01.245962 26473 raft_consensus.cc:729] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5a4dc3c93b4e09a375027c6e7854d9, State: Initialized, Role: FOLLOWER
I20260810 06:50:01.245987 26479 tablet_bootstrap.cc:492] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: No bootstrap required, opened a new log
I20260810 06:50:01.246127 26479 ts_tablet_manager.cc:1397] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260810 06:50:01.246217 26473 consensus_queue.cc:260] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.247092 26473 ts_tablet_manager.cc:1428] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:50:01.247027 26479 raft_consensus.cc:348] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.247500 26479 raft_consensus.cc:374] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:01.247661 26479 raft_consensus.cc:729] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Initialized, Role: FOLLOWER
I20260810 06:50:01.247937 26479 consensus_queue.cc:260] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.248687 26479 ts_tablet_manager.cc:1428] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 06:50:01.250435 26476 tablet_bootstrap.cc:492] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: No bootstrap required, opened a new log
I20260810 06:50:01.250675 26476 ts_tablet_manager.cc:1397] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260810 06:50:01.251526 26476 raft_consensus.cc:348] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.251787 26476 raft_consensus.cc:374] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:01.251901 26476 raft_consensus.cc:729] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01b9ac4b73441a58d8267d21edc3c26, State: Initialized, Role: FOLLOWER
I20260810 06:50:01.252156 26476 consensus_queue.cc:260] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.253135 26476 ts_tablet_manager.cc:1428] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 06:50:01.354533 26391 raft_consensus.cc:482] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:01.354728 26391 raft_consensus.cc:504] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.355381 26391 leader_election.cc:288] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225)
I20260810 06:50:01.355970 25744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e059c57c9ad840cc9e6af75e4566ca05" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26" is_pre_election: true
I20260810 06:50:01.356005 25988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e059c57c9ad840cc9e6af75e4566ca05" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" is_pre_election: true
I20260810 06:50:01.356258 25988 raft_consensus.cc:2454] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:50:01.356245 25744 raft_consensus.cc:2454] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 0.
I20260810 06:50:01.356802 26106 leader_election.cc:302] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, cd5a4dc3c93b4e09a375027c6e7854d9, f01b9ac4b73441a58d8267d21edc3c26; no voters:
I20260810 06:50:01.357085 26391 raft_consensus.cc:2781] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:01.357189 26391 raft_consensus.cc:482] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:01.357255 26391 raft_consensus.cc:3037] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:01.359216 26391 raft_consensus.cc:504] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.359719 26391 leader_election.cc:288] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Requested vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225)
I20260810 06:50:01.360172 25744 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e059c57c9ad840cc9e6af75e4566ca05" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26"
I20260810 06:50:01.360229 25988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e059c57c9ad840cc9e6af75e4566ca05" candidate_uuid: "4347f2a8842642508eeb34f496ba404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9"
I20260810 06:50:01.360356 25744 raft_consensus.cc:3037] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:01.360414 25988 raft_consensus.cc:3037] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:01.362375 25744 raft_consensus.cc:2454] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:50:01.362481 25988 raft_consensus.cc:2454] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4347f2a8842642508eeb34f496ba404a in term 1.
I20260810 06:50:01.362803 26100 leader_election.cc:302] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4347f2a8842642508eeb34f496ba404a, f01b9ac4b73441a58d8267d21edc3c26; no voters:
I20260810 06:50:01.363039 26391 raft_consensus.cc:2781] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:01.363178 26391 raft_consensus.cc:686] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 1 LEADER]: Becoming Leader. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Running, Role: LEADER
I20260810 06:50:01.363446 26391 consensus_queue.cc:237] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.365860 25463 catalog_manager.cc:5495] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a reported cstate change: term changed from 0 to 1, leader changed from <none> to 4347f2a8842642508eeb34f496ba404a (127.22.11.131). New cstate: current_term: 1 leader_uuid: "4347f2a8842642508eeb34f496ba404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } health_report { overall_health: HEALTHY } } }
I20260810 06:50:01.402534 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.396952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35618 (local address 127.22.11.131:39083)
0810 06:50:01.397546 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:01.397551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:01.397568 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:50:01.397614 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:01.397617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:01.397679 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:01.397762 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:01.398178 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.399139 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.400393 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.400825 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.400930 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.401559 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:01.401749 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:01.401825 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:01.401971 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:50:01.402045 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":339,"threads_started":1}
I20260810 06:50:01.412180 25744 raft_consensus.cc:1261] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Refusing update from remote peer 4347f2a8842642508eeb34f496ba404a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:01.412137 25988 raft_consensus.cc:1261] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Refusing update from remote peer 4347f2a8842642508eeb34f496ba404a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:01.412861 26494 consensus_queue.cc:1035] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:01.413254 26391 consensus_queue.cc:1035] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:01.457647 25464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:01.457974 25464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:01.461004 25464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Sending DeleteTablet for 3 replicas of tablet e059c57c9ad840cc9e6af75e4566ca05
I20260810 06:50:01.461627 25712 tablet_service.cc:1514] Processing DeleteTablet for tablet e059c57c9ad840cc9e6af75e4566ca05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:01 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:36360
I20260810 06:50:01.461877 25956 tablet_service.cc:1514] Processing DeleteTablet for tablet e059c57c9ad840cc9e6af75e4566ca05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:01 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50138
I20260810 06:50:01.461932 26446 tablet_replica.cc:330] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: stopping tablet replica
I20260810 06:50:01.461988 26171 tablet_service.cc:1514] Processing DeleteTablet for tablet e059c57c9ad840cc9e6af75e4566ca05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:01 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 06:50:01.462205 26448 tablet_replica.cc:330] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: stopping tablet replica
I20260810 06:50:01.462230 26446 raft_consensus.cc:2229] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:01.462417 26446 raft_consensus.cc:2258] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:01.462509 26449 tablet_replica.cc:330] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: stopping tablet replica
I20260810 06:50:01.462489 26448 raft_consensus.cc:2229] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:01.462769 26448 raft_consensus.cc:2258] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:01.462811 26449 raft_consensus.cc:2229] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:01.463227 26449 raft_consensus.cc:2258] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:01.463183 25464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45550:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:01.463981 25464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:01.464186 26449 ts_tablet_manager.cc:1905] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:01.466161 26446 ts_tablet_manager.cc:1905] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:01.469460 26448 ts_tablet_manager.cc:1905] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:01.474296 26446 ts_tablet_manager.cc:1918] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:01.474567 26446 log.cc:1196] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/e059c57c9ad840cc9e6af75e4566ca05
I20260810 06:50:01.474743 26449 ts_tablet_manager.cc:1918] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:01.474905 26449 log.cc:1196] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/e059c57c9ad840cc9e6af75e4566ca05
I20260810 06:50:01.475126 26446 ts_tablet_manager.cc:1939] T e059c57c9ad840cc9e6af75e4566ca05 P f01b9ac4b73441a58d8267d21edc3c26: Deleting consensus metadata
I20260810 06:50:01.475651 26449 ts_tablet_manager.cc:1939] T e059c57c9ad840cc9e6af75e4566ca05 P 4347f2a8842642508eeb34f496ba404a: Deleting consensus metadata
I20260810 06:50:01.476236 25437 catalog_manager.cc:4841] TS f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413): tablet e059c57c9ad840cc9e6af75e4566ca05 (table test-table [id=a4a8121e8d2047099c6c61c59a5a77b2]) successfully deleted
I20260810 06:50:01.477952 26448 ts_tablet_manager.cc:1918] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:01.478193 26448 log.cc:1196] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/e059c57c9ad840cc9e6af75e4566ca05
I20260810 06:50:01.478564 25440 catalog_manager.cc:4841] TS 4347f2a8842642508eeb34f496ba404a (127.22.11.131:39083): tablet e059c57c9ad840cc9e6af75e4566ca05 (table test-table [id=a4a8121e8d2047099c6c61c59a5a77b2]) successfully deleted
I20260810 06:50:01.479198 26448 ts_tablet_manager.cc:1939] T e059c57c9ad840cc9e6af75e4566ca05 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting consensus metadata
I20260810 06:50:01.480911 25439 catalog_manager.cc:4841] TS cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225): tablet e059c57c9ad840cc9e6af75e4566ca05 (table test-table [id=a4a8121e8d2047099c6c61c59a5a77b2]) successfully deleted
I20260810 06:50:01.480856 25712 tablet_service.cc:1467] Processing CreateTablet for tablet 426017c0411c49cfbfdcbce11b207138 (DEFAULT_TABLE table=test-table [id=37bf77f86fb14f10bf972d7da67a0762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:01.481156 26171 tablet_service.cc:1467] Processing CreateTablet for tablet 426017c0411c49cfbfdcbce11b207138 (DEFAULT_TABLE table=test-table [id=37bf77f86fb14f10bf972d7da67a0762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:01.481408 25712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426017c0411c49cfbfdcbce11b207138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:01.481614 26171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426017c0411c49cfbfdcbce11b207138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:01.481876 25956 tablet_service.cc:1467] Processing CreateTablet for tablet 426017c0411c49cfbfdcbce11b207138 (DEFAULT_TABLE table=test-table [id=37bf77f86fb14f10bf972d7da67a0762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:01.482331 25956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 426017c0411c49cfbfdcbce11b207138. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:01.489261 26479 tablet_bootstrap.cc:492] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: Bootstrap starting.
I20260810 06:50:01.489360 26476 tablet_bootstrap.cc:492] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: Bootstrap starting.
I20260810 06:50:01.489461 26473 tablet_bootstrap.cc:492] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: Bootstrap starting.
I20260810 06:50:01.491042 26479 tablet_bootstrap.cc:654] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:01.491343 26473 tablet_bootstrap.cc:654] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:01.491628 26476 tablet_bootstrap.cc:654] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:01.506079 26473 tablet_bootstrap.cc:492] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: No bootstrap required, opened a new log
I20260810 06:50:01.507313 26473 ts_tablet_manager.cc:1397] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.002s
I20260810 06:50:01.508421 26473 raft_consensus.cc:348] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.508833 26473 raft_consensus.cc:374] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:01.508987 26473 raft_consensus.cc:729] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5a4dc3c93b4e09a375027c6e7854d9, State: Initialized, Role: FOLLOWER
I20260810 06:50:01.509346 26473 consensus_queue.cc:260] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.510057 26473 ts_tablet_manager.cc:1428] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:01.510367 26476 tablet_bootstrap.cc:492] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: No bootstrap required, opened a new log
I20260810 06:50:01.510661 26476 ts_tablet_manager.cc:1397] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.010s
I20260810 06:50:01.512758 26479 tablet_bootstrap.cc:492] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: No bootstrap required, opened a new log
I20260810 06:50:01.512995 26479 ts_tablet_manager.cc:1397] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.013s
I20260810 06:50:01.513214 26476 raft_consensus.cc:348] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.513706 26476 raft_consensus.cc:374] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:01.513863 26476 raft_consensus.cc:729] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01b9ac4b73441a58d8267d21edc3c26, State: Initialized, Role: FOLLOWER
I20260810 06:50:01.514017 26479 raft_consensus.cc:348] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.514372 26479 raft_consensus.cc:374] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:01.514503 26479 raft_consensus.cc:729] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4347f2a8842642508eeb34f496ba404a, State: Initialized, Role: FOLLOWER
I20260810 06:50:01.514333 26476 consensus_queue.cc:260] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.514775 26479 consensus_queue.cc:260] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.515254 26479 ts_tablet_manager.cc:1428] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 06:50:01.515874 26476 ts_tablet_manager.cc:1428] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260810 06:50:01.558848 26488 raft_consensus.cc:482] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:01.559217 26488 raft_consensus.cc:504] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.561167 26488 leader_election.cc:288] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), 4347f2a8842642508eeb34f496ba404a (127.22.11.131:39083)
I20260810 06:50:01.571352 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.561676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:47853 (local address 127.22.11.129:46413)
0810 06:50:01.562720 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:01.562727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:01.563255 (+ 528us) server_negotiation.cc:407] Connection header received
0810 06:50:01.563522 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:01.563526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:01.563587 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:01.563675 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:01.565041 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.566207 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.568878 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.570050 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.570273 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.570365 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:01.570638 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 06:50:01.570769 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":614,"threads_started":1}
I20260810 06:50:01.571403 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.561599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:46413 (local address 127.22.11.130:47853)
0810 06:50:01.563149 (+ 1550us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:01.563165 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:01.563347 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:01.563713 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:01.563718 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:01.563792 (+ 74us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:01.564766 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.564780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.566477 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.566481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.568727 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.568741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.570327 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.570334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.570488 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.570514 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:50:01.570605 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 06:50:01.570742 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1409,"mutex_wait_us":25,"thread_start_us":1122,"threads_started":1}
I20260810 06:50:01.571403 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.561435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:49845 (local address 127.22.11.131:39083)
0810 06:50:01.561594 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:01.561599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:01.563836 (+ 2237us) server_negotiation.cc:407] Connection header received
0810 06:50:01.564043 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:01.564047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:01.564102 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:01.564173 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:01.565425 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.566501 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.569236 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.570409 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.570559 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.570632 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:01.570986 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 06:50:01.571118 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:50:01.571899 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.561807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39083 (local address 127.22.11.130:49845)
0810 06:50:01.563738 (+ 1931us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:01.563752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:01.563917 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:01.564208 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:01.564212 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:01.564235 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:01.565260 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.565272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.566743 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.566749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.569082 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.569096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.570656 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.570663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.570822 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.570848 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:50:01.570935 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:50:01.571060 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1799,"mutex_wait_us":6,"thread_start_us":1523,"threads_started":1}
I20260810 06:50:01.572239 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426017c0411c49cfbfdcbce11b207138" candidate_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26" is_pre_election: true
I20260810 06:50:01.572604 25745 raft_consensus.cc:2454] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd5a4dc3c93b4e09a375027c6e7854d9 in term 0.
I20260810 06:50:01.573094 26211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426017c0411c49cfbfdcbce11b207138" candidate_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4347f2a8842642508eeb34f496ba404a" is_pre_election: true
I20260810 06:50:01.573241 25869 leader_election.cc:302] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd5a4dc3c93b4e09a375027c6e7854d9, f01b9ac4b73441a58d8267d21edc3c26; no voters:
I20260810 06:50:01.573542 26211 raft_consensus.cc:2454] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd5a4dc3c93b4e09a375027c6e7854d9 in term 0.
I20260810 06:50:01.573724 26488 raft_consensus.cc:2781] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:01.573946 26488 raft_consensus.cc:482] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:01.574116 26488 raft_consensus.cc:3037] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:01.576840 26488 raft_consensus.cc:504] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.577603 26488 leader_election.cc:288] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [CANDIDATE]: Term 1 election: Requested vote from peers f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413), 4347f2a8842642508eeb34f496ba404a (127.22.11.131:39083)
I20260810 06:50:01.577919 25745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426017c0411c49cfbfdcbce11b207138" candidate_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01b9ac4b73441a58d8267d21edc3c26"
I20260810 06:50:01.578141 25745 raft_consensus.cc:3037] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:01.578119 26211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "426017c0411c49cfbfdcbce11b207138" candidate_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4347f2a8842642508eeb34f496ba404a"
I20260810 06:50:01.578280 26211 raft_consensus.cc:3037] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:01.580653 25745 raft_consensus.cc:2454] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd5a4dc3c93b4e09a375027c6e7854d9 in term 1.
I20260810 06:50:01.581002 25869 leader_election.cc:302] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd5a4dc3c93b4e09a375027c6e7854d9, f01b9ac4b73441a58d8267d21edc3c26; no voters:
I20260810 06:50:01.581198 26211 raft_consensus.cc:2454] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd5a4dc3c93b4e09a375027c6e7854d9 in term 1.
I20260810 06:50:01.581367 26488 raft_consensus.cc:2781] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:01.582018 26488 raft_consensus.cc:686] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 LEADER]: Becoming Leader. State: Replica: cd5a4dc3c93b4e09a375027c6e7854d9, State: Running, Role: LEADER
I20260810 06:50:01.582520 26488 consensus_queue.cc:237] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } }
I20260810 06:50:01.586154 25464 catalog_manager.cc:5495] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130). New cstate: current_term: 1 leader_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd5a4dc3c93b4e09a375027c6e7854d9" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 41225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 } health_report { overall_health: UNKNOWN } } }
I20260810 06:50:01.611177 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.604839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57310 (local address 127.22.11.130:41225)
0810 06:50:01.605687 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:01.605691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:01.605709 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:50:01.605784 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:01.605787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:01.605869 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:01.606013 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:01.606436 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.607292 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.608996 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.609407 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.609499 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.609876 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:01.610512 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:01.610624 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:01.610768 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 06:50:01.610861 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":581,"threads_started":1}
I20260810 06:50:01.625696 26492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.620579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:48153 (local address 127.22.11.131:39083)
0810 06:50:01.620762 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:01.620768 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:01.621390 (+ 622us) server_negotiation.cc:407] Connection header received
0810 06:50:01.621646 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:01.621650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:01.621716 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:01.621787 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:01.622164 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.622929 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.624276 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.624700 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.624793 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.625171 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:01.625262 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:01.625316 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:01.625419 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 06:50:01.625480 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:50:01.625860 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.620598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39083 (local address 127.22.11.130:48153)
0810 06:50:01.621223 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:01.621238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:01.621452 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:01.621807 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:01.621810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:01.621820 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:50:01.622058 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.622065 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.623036 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.623039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.624165 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.624177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.624848 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.624852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.624979 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.625039 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:50:01.625323 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:50:01.625329 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:50:01.625379 (+ 50us) client_negotiation.cc:240] Negotiation successful
0810 06:50:01.625470 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":299,"threads_started":1}
I20260810 06:50:01.633793 25745 raft_consensus.cc:1261] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Refusing update from remote peer cd5a4dc3c93b4e09a375027c6e7854d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:01.634081 26211 raft_consensus.cc:1261] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Refusing update from remote peer cd5a4dc3c93b4e09a375027c6e7854d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:01.634496 26506 consensus_queue.cc:1035] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01b9ac4b73441a58d8267d21edc3c26" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 46413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:01.634910 26488 consensus_queue.cc:1035] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4347f2a8842642508eeb34f496ba404a" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:01.660383 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.654970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:35229 (local address 127.22.11.130:41225)
0810 06:50:01.655143 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:01.655147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:01.655858 (+ 711us) server_negotiation.cc:407] Connection header received
0810 06:50:01.655992 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:01.655995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:01.656055 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:01.656124 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:01.656529 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.657291 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.658873 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.659269 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.659381 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.659738 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:01.659832 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:01.659903 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:01.660036 (+ 133us) server_negotiation.cc:299] Negotiation successful
0810 06:50:01.660111 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:50:01.660480 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:01.654854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:41225 (local address 127.22.11.131:35229)
0810 06:50:01.655694 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:01.655709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:01.655906 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:01.656137 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:01.656140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:01.656149 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:50:01.656420 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.656430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.657432 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.657438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.658740 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:01.658753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:01.659440 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:01.659447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:01.659567 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:01.659603 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:50:01.659930 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:50:01.659941 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 06:50:01.660007 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 06:50:01.660135 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":579,"threads_started":1}
W20260810 06:50:01.661487 26419 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 06:50:03.253897 25464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:03.254151 25464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45550:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:03.260883 25464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a851e9f0d9249ca9eb8b61814e78902: Sending DeleteTablet for 3 replicas of tablet 426017c0411c49cfbfdcbce11b207138
I20260810 06:50:03.261823 25712 tablet_service.cc:1514] Processing DeleteTablet for tablet 426017c0411c49cfbfdcbce11b207138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:36360
I20260810 06:50:03.261808 25956 tablet_service.cc:1514] Processing DeleteTablet for tablet 426017c0411c49cfbfdcbce11b207138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50138
I20260810 06:50:03.262164 26171 tablet_service.cc:1514] Processing DeleteTablet for tablet 426017c0411c49cfbfdcbce11b207138 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 06:50:03.263438 26565 tablet_replica.cc:330] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: stopping tablet replica
I20260810 06:50:03.263734 26565 raft_consensus.cc:2229] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:03.264182 26565 raft_consensus.cc:2258] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:03.264134 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25593
I20260810 06:50:03.265415 26565 ts_tablet_manager.cc:1905] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:03.274166 26565 ts_tablet_manager.cc:1918] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:50:03.274382 26565 log.cc:1196] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/426017c0411c49cfbfdcbce11b207138
I20260810 06:50:03.274569 26564 tablet_replica.cc:330] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: stopping tablet replica
I20260810 06:50:03.274780 26564 raft_consensus.cc:2229] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:03.274868 26565 ts_tablet_manager.cc:1939] T 426017c0411c49cfbfdcbce11b207138 P 4347f2a8842642508eeb34f496ba404a: Deleting consensus metadata
I20260810 06:50:03.275065 26564 raft_consensus.cc:2258] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:03.276271 25440 catalog_manager.cc:4841] TS 4347f2a8842642508eeb34f496ba404a (127.22.11.131:39083): tablet 426017c0411c49cfbfdcbce11b207138 (table test-table [id=37bf77f86fb14f10bf972d7da67a0762]) successfully deleted
I20260810 06:50:03.276748 26564 ts_tablet_manager.cc:1905] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:03.278658 26563 tablet_replica.cc:330] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: stopping tablet replica
I20260810 06:50:03.278882 26563 raft_consensus.cc:2229] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:03.279489 26563 raft_consensus.cc:2258] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:03.280512 26563 ts_tablet_manager.cc:1905] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:50:03 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:03.281656 26564 ts_tablet_manager.cc:1918] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:50:03.281782 26564 log.cc:1196] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/426017c0411c49cfbfdcbce11b207138
I20260810 06:50:03.282228 26564 ts_tablet_manager.cc:1939] T 426017c0411c49cfbfdcbce11b207138 P f01b9ac4b73441a58d8267d21edc3c26: Deleting consensus metadata
I20260810 06:50:03.283329 25437 catalog_manager.cc:4841] TS f01b9ac4b73441a58d8267d21edc3c26 (127.22.11.129:46413): tablet 426017c0411c49cfbfdcbce11b207138 (table test-table [id=37bf77f86fb14f10bf972d7da67a0762]) successfully deleted
I20260810 06:50:03.285966 26563 ts_tablet_manager.cc:1918] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 06:50:03.286111 26563 log.cc:1196] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/426017c0411c49cfbfdcbce11b207138
I20260810 06:50:03.286523 26563 ts_tablet_manager.cc:1939] T 426017c0411c49cfbfdcbce11b207138 P cd5a4dc3c93b4e09a375027c6e7854d9: Deleting consensus metadata
I20260810 06:50:03.287462 25439 catalog_manager.cc:4841] TS cd5a4dc3c93b4e09a375027c6e7854d9 (127.22.11.130:41225): tablet 426017c0411c49cfbfdcbce11b207138 (table test-table [id=37bf77f86fb14f10bf972d7da67a0762]) successfully deleted
I20260810 06:50:03.288164 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:03.267135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57196 (local address 127.22.11.129:46413)
0810 06:50:03.267859 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:03.267864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:03.267886 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:50:03.267946 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:03.267952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:03.268034 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:03.270353 (+ 2319us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:03.270623 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:03.272251 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:03.274349 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:03.275133 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:03.275261 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:03.282051 (+ 6790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:03.282070 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:03.282091 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:03.282132 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:03.285342 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:03.286111 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:03.286135 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:03.286139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:03.286297 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:03.286700 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:03.286704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:03.286706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:03.287068 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:03.287275 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:03.287680 (+ 405us) server_negotiation.cc:299] Negotiation successful
0810 06:50:03.287851 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":411,"threads_started":1}
W20260810 06:50:03.288834 25831 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57196
W20260810 06:50:03.289409 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25593: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:50:03.289628 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25593
I20260810 06:50:03.319250 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25846
Aug 10 06:50:03 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:50:03.334394 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:03.320016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.22.11.130:41225)
0810 06:50:03.320573 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:03.320577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:03.320593 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:50:03.320678 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:03.320682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:03.320744 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:03.320838 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:03.322195 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:03.322984 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:03.324285 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:03.324780 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:03.324892 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:03.328860 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:03.328875 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:03.328890 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:03.328927 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:03.331685 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:03.332309 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:03.332317 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:03.332319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:03.332485 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:03.332871 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:03.332875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:03.332877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:03.333251 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:03.333520 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:03.333845 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 06:50:03.334068 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":282,"threads_started":1}
W20260810 06:50:03.335064 26062 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57312
W20260810 06:50:03.335661 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25846: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:50:03.335824 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25846
I20260810 06:50:03.366335 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26076
Aug 10 06:50:03 dist-test-slave-f7th krb5kdc[25385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344601, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:50:03.380893 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:03.367143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.22.11.131:39083)
0810 06:50:03.367635 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:03.367640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:03.367655 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:50:03.367709 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:03.367713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:03.367771 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:03.367865 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:03.369107 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:03.369815 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:03.370944 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:03.371313 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:03.371400 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:03.375477 (+ 4077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:03.375492 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:03.375511 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:03.375543 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:03.378147 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:03.378772 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:03.378779 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:03.378782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:03.378907 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:03.379256 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:03.379260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:03.379262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:03.379618 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:03.379912 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:03.380184 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 06:50:03.380395 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":236,"threads_started":1}
W20260810 06:50:03.381503 26293 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35628
W20260810 06:50:03.382053 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26076: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:50:03.382215 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26076
I20260810 06:50:03.424808 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25412
I20260810 06:50:03.425959 25538 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:03.544098 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25412
2026-08-10T06:50:03Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8477 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:03 dist-test-slave-f7th krb5kdc[26579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:03 dist-test-slave-f7th krb5kdc[26579](info): set up 2 sockets
Aug 10 06:50:03 dist-test-slave-f7th krb5kdc[26579](info): commencing operation
krb5kdc: starting...
W20260810 06:50:04.779376 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:04 dist-test-slave-f7th krb5kdc[26579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:04.970286 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39269
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:39269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:05.287789 26600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:05.288082 26600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:05.288210 26600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:05.295169 26600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:05.295276 26600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:05.295351 26600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:05.295424 26600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:05.295483 26600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:05.304880 26600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:39269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:05.306149 26600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:05.307439 26600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:05.318647 26611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:05.318599 26614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:05.318732 26612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:05.319494 26600 server_base.cc:1034] running on GCE node
I20260810 06:50:05.320219 26600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:05.321576 26600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:05.322765 26600 hybrid_clock.cc:648] HybridClock initialized: now 1786344605322723 us; error 29 us; skew 500 ppm
I20260810 06:50:05.323086 26600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:05 dist-test-slave-f7th krb5kdc[26579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344605, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:05.326069 26600 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:05.327477 26600 webserver.cc:460] Webserver started at http://127.22.11.190:41935/ using document root <none> and password file <none>
I20260810 06:50:05.328114 26600 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:05.328231 26600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:05.328533 26600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:05.331401 26600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "ef1c5a569199481b838eb7d397ea0edf"
format_stamp: "Formatted at 2026-08-10 06:50:05 on dist-test-slave-f7th"
server_key: "847af23e917f67585c461cf34798e636"
server_key_iv: "5234880f6f8a7e94a3b3c84463dc0c39"
server_key_version: "encryptionkey@0"
I20260810 06:50:05.332119 26600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "ef1c5a569199481b838eb7d397ea0edf"
format_stamp: "Formatted at 2026-08-10 06:50:05 on dist-test-slave-f7th"
server_key: "847af23e917f67585c461cf34798e636"
server_key_iv: "5234880f6f8a7e94a3b3c84463dc0c39"
server_key_version: "encryptionkey@0"
I20260810 06:50:05.337093 26600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 06:50:05.340387 26630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:05.341769 26600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:05.341936 26600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "ef1c5a569199481b838eb7d397ea0edf"
format_stamp: "Formatted at 2026-08-10 06:50:05 on dist-test-slave-f7th"
server_key: "847af23e917f67585c461cf34798e636"
server_key_iv: "5234880f6f8a7e94a3b3c84463dc0c39"
server_key_version: "encryptionkey@0"
I20260810 06:50:05.342121 26600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:05.371797 26600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:05.375538 26600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:05.375802 26600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:05.392302 26600 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:39269
I20260810 06:50:05.392308 26718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:39269 every 8 connection(s)
I20260810 06:50:05.393654 26600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:05.395694 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26600
I20260810 06:50:05.395936 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:05.396760 22574 external_mini_cluster.cc:1428] Setting key ae50d814bb554d72766c36d96db2cc1c
I20260810 06:50:05.397894 26720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:05.410715 26720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf: Bootstrap starting.
Aug 10 06:50:05 dist-test-slave-f7th krb5kdc[26579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:05.414553 26720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:05.415843 26720 log.cc:824] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:05.419039 26720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf: No bootstrap required, opened a new log
I20260810 06:50:05.419927 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:05.398720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.22.11.190:39269)
0810 06:50:05.399489 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:05.399515 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:05.399567 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:50:05.401154 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:05.401175 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:05.401543 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:05.401911 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:05.403343 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:05.404706 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:05.406479 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:05.407245 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:05.407394 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:05.411810 (+ 4416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:05.411822 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:05.411839 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:05.411888 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:05.415607 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:05.416268 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:05.416278 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:05.416290 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:05.416511 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:05.416947 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:05.416950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:05.416952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:05.417593 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:05.417870 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:05.418195 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 06:50:05.418592 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":342,"threads_started":1}
I20260810 06:50:05.426580 26720 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1c5a569199481b838eb7d397ea0edf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39269 } }
I20260810 06:50:05.426906 26720 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:05.426978 26720 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1c5a569199481b838eb7d397ea0edf, State: Initialized, Role: FOLLOWER
I20260810 06:50:05.427431 26720 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1c5a569199481b838eb7d397ea0edf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39269 } }
I20260810 06:50:05.427625 26720 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:05.427719 26720 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:05.427839 26720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:05.429955 26720 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1c5a569199481b838eb7d397ea0edf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39269 } }
I20260810 06:50:05.430392 26720 leader_election.cc:302] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef1c5a569199481b838eb7d397ea0edf; no voters:
I20260810 06:50:05.430927 26720 leader_election.cc:288] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:05.431054 26727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:05.431658 26727 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [term 1 LEADER]: Becoming Leader. State: Replica: ef1c5a569199481b838eb7d397ea0edf, State: Running, Role: LEADER
I20260810 06:50:05.432128 26727 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1c5a569199481b838eb7d397ea0edf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39269 } }
I20260810 06:50:05.432809 26720 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:05.434598 26730 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef1c5a569199481b838eb7d397ea0edf. Latest consensus state: current_term: 1 leader_uuid: "ef1c5a569199481b838eb7d397ea0edf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1c5a569199481b838eb7d397ea0edf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39269 } } }
I20260810 06:50:05.435029 26730 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:05.434623 26728 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef1c5a569199481b838eb7d397ea0edf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef1c5a569199481b838eb7d397ea0edf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39269 } } }
I20260810 06:50:05.436123 26728 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:05.441998 26737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:05.446689 26737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:05.455572 26737 catalog_manager.cc:1347] Generated new cluster ID: 26bdaed727154f67a1d1cee00d3e389e
I20260810 06:50:05.455686 26737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:05.476877 26737 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:05.478089 26737 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:05.486243 26737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef1c5a569199481b838eb7d397ea0edf: Generated new TSK 0
I20260810 06:50:05.486999 26737 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:50:05.549909 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39269
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:05.868352 26759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:05.868651 26759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:05.868793 26759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:05.876259 26759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:05.876358 26759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:05.876497 26759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:05.885936 26759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:05.887215 26759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:05.888541 26759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:05.901149 26767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:05.901631 26768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:05.901589 26770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:05.902058 26759 server_base.cc:1034] running on GCE node
I20260810 06:50:05.902689 26759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:05.903599 26759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:05.904794 26759 hybrid_clock.cc:648] HybridClock initialized: now 1786344605904756 us; error 30 us; skew 500 ppm
I20260810 06:50:05.905087 26759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:05 dist-test-slave-f7th krb5kdc[26579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344605, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:05.908154 26759 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:50:05.909518 26759 webserver.cc:460] Webserver started at http://127.22.11.129:45945/ using document root <none> and password file <none>
I20260810 06:50:05.910151 26759 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:05.910259 26759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:05.910564 26759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:05.913393 26759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "7c9e30c8110a4d628e426f4fc6209b40"
format_stamp: "Formatted at 2026-08-10 06:50:05 on dist-test-slave-f7th"
server_key: "cad5ec99f93d006ac1f2dcebaa4ba7b6"
server_key_iv: "df96a0138a3c401377dc0c459f62b19b"
server_key_version: "encryptionkey@0"
I20260810 06:50:05.914130 26759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "7c9e30c8110a4d628e426f4fc6209b40"
format_stamp: "Formatted at 2026-08-10 06:50:05 on dist-test-slave-f7th"
server_key: "cad5ec99f93d006ac1f2dcebaa4ba7b6"
server_key_iv: "df96a0138a3c401377dc0c459f62b19b"
server_key_version: "encryptionkey@0"
I20260810 06:50:05.918964 26759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:50:05.922194 26789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:05.923359 26759 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:50:05.923516 26759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "7c9e30c8110a4d628e426f4fc6209b40"
format_stamp: "Formatted at 2026-08-10 06:50:05 on dist-test-slave-f7th"
server_key: "cad5ec99f93d006ac1f2dcebaa4ba7b6"
server_key_iv: "df96a0138a3c401377dc0c459f62b19b"
server_key_version: "encryptionkey@0"
I20260810 06:50:05.923681 26759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:05.944456 26759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:05.949055 26759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:05.949326 26759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:05.950316 26759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:05.953313 26759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:05.953388 26759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:05.953510 26759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:05.953568 26759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:05.985473 26759 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:39569
I20260810 06:50:05.985589 26989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:39569 every 8 connection(s)
I20260810 06:50:05.986752 26759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:05.988116 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26759
I20260810 06:50:05.988351 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:05.989068 22574 external_mini_cluster.cc:1428] Setting key e0ffc6b3d3172a40ebd8f6c180618d9c
Aug 10 06:50:05 dist-test-slave-f7th krb5kdc[26579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344605, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:06.010579 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:05.989924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:43217 (local address 127.22.11.190:39269)
0810 06:50:05.990355 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:05.990359 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:05.991683 (+ 1324us) server_negotiation.cc:407] Connection header received
0810 06:50:05.994175 (+ 2492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:05.994179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:05.994250 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:05.994344 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:05.996225 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:05.996995 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:05.998985 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:05.999384 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:05.999512 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:06.003738 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:06.003745 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:06.003750 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:06.003790 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:06.007023 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:06.007848 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:06.007855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:06.007858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:06.008020 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:06.008579 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:06.008584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:06.008587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:06.009006 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:06.009163 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:06.009935 (+ 772us) server_negotiation.cc:299] Negotiation successful
0810 06:50:06.010192 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":238,"threads_started":1}
I20260810 06:50:06.011390 26993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:05.990630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39269 (local address 127.22.11.129:43217)
0810 06:50:05.991513 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:05.991567 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:05.993860 (+ 2293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:05.994597 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:05.994607 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:05.995289 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:05.996062 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:05.996088 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:05.997141 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:05.997145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:05.998849 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:05.998864 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:05.999615 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:05.999623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:05.999866 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:06.000911 (+ 1045us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:06.000947 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:06.003482 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:06.007178 (+ 3696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:06.007188 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:06.007206 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:06.007713 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:06.008166 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:06.008170 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:06.008172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:06.008434 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:06.009134 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:06.009141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:06.009500 (+ 359us) client_negotiation.cc:769] Sending connection context
0810 06:50:06.009781 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 06:50:06.010229 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":338,"threads_started":1}
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
I20260810 06:50:06.013396 26990 heartbeater.cc:347] Connected to a master server at 127.22.11.190:39269
I20260810 06:50:06.013692 26990 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:06.014425 26990 heartbeater.cc:511] Master 127.22.11.190:39269 requested a full tablet report, sending...
I20260810 06:50:06.016503 26656 ts_manager.cc:194] Registered new tserver with Master: 7c9e30c8110a4d628e426f4fc6209b40 (127.22.11.129:39569)
I20260810 06:50:06.018399 26656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:43217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
W20260810 06:50:06.662281 22574 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.11.130: real 0.670s user 0.004s sys 0.012s
I20260810 06:50:06.662984 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39269
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:06.990700 26997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:06.990960 26997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:06.991103 26997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:06.998193 26997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:06.998304 26997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:06.998459 26997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:07.008056 26997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:07.009434 26997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:07.010725 26997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:50:07.021273 26990 heartbeater.cc:503] Master 127.22.11.190:39269 was elected leader, sending a full tablet report...
W20260810 06:50:07.023679 27006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:07.023813 27010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:07.024267 27005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:07.024762 26997 server_base.cc:1034] running on GCE node
I20260810 06:50:07.025358 26997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:07.026333 26997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:07.027554 26997 hybrid_clock.cc:648] HybridClock initialized: now 1786344607027483 us; error 43 us; skew 500 ppm
I20260810 06:50:07.027879 26997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:07 dist-test-slave-f7th krb5kdc[26579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344607, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:07.035583 26997 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:50:07.037118 26997 webserver.cc:460] Webserver started at http://127.22.11.130:35329/ using document root <none> and password file <none>
I20260810 06:50:07.037772 26997 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:07.037868 26997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:07.038131 26997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:07.041093 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "6f428203307e446da544161e95244757"
format_stamp: "Formatted at 2026-08-10 06:50:07 on dist-test-slave-f7th"
server_key: "8d584eacb0f33a66e2f443dd0123b0f5"
server_key_iv: "b338d4b453112ac36ead693d203b9fb7"
server_key_version: "encryptionkey@0"
I20260810 06:50:07.041811 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "6f428203307e446da544161e95244757"
format_stamp: "Formatted at 2026-08-10 06:50:07 on dist-test-slave-f7th"
server_key: "8d584eacb0f33a66e2f443dd0123b0f5"
server_key_iv: "b338d4b453112ac36ead693d203b9fb7"
server_key_version: "encryptionkey@0"
I20260810 06:50:07.046959 26997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:50:07.050226 27027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:07.051463 26997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:07.051596 26997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "6f428203307e446da544161e95244757"
format_stamp: "Formatted at 2026-08-10 06:50:07 on dist-test-slave-f7th"
server_key: "8d584eacb0f33a66e2f443dd0123b0f5"
server_key_iv: "b338d4b453112ac36ead693d203b9fb7"
server_key_version: "encryptionkey@0"
I20260810 06:50:07.051749 26997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:07.065831 26997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:07.069461 26997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:07.069694 26997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:07.070565 26997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:07.072032 26997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:07.072108 26997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:07.072202 26997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:07.072248 26997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:07.102306 26997 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:46551
I20260810 06:50:07.102378 27225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:46551 every 8 connection(s)
I20260810 06:50:07.103587 26997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:07.112697 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26997
I20260810 06:50:07.112855 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:07.113292 22574 external_mini_cluster.cc:1428] Setting key a77264869ad9104cc8de69f72b099adf
Aug 10 06:50:07 dist-test-slave-f7th krb5kdc[26579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344607, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:07.123217 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:07.106090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:50017 (local address 127.22.11.190:39269)
0810 06:50:07.106523 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:07.106527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:07.107390 (+ 863us) server_negotiation.cc:407] Connection header received
0810 06:50:07.109298 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:07.109304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:07.109389 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:07.109486 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:07.111013 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:07.111721 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:07.113337 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:07.113800 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:07.113907 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:07.117678 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:07.117683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:07.117687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:07.117718 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:07.120134 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:07.121030 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:07.121035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:07.121037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:07.121155 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:07.121572 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:07.121575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:07.121577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:07.121819 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:07.121927 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:07.122675 (+ 748us) server_negotiation.cc:299] Negotiation successful
0810 06:50:07.122855 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":223,"threads_started":1}
I20260810 06:50:07.124169 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:07.106510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39269 (local address 127.22.11.130:50017)
0810 06:50:07.107221 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:07.107258 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:07.109017 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:07.109741 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:07.109750 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:07.110255 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:07.110828 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:07.110846 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:07.111861 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:07.111865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:07.113197 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:07.113210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:07.114031 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:07.114039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:07.114252 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:07.114903 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:07.114925 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:07.117455 (+ 2530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:07.120272 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:07.120281 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:07.120296 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:07.120890 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:07.121248 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:07.121252 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:07.121255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:07.121482 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:07.121940 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:07.121947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:07.122327 (+ 380us) client_negotiation.cc:769] Sending connection context
0810 06:50:07.122601 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 06:50:07.123073 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":273,"threads_started":1}
I20260810 06:50:07.125492 27227 heartbeater.cc:347] Connected to a master server at 127.22.11.190:39269
I20260810 06:50:07.125770 27227 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:07.126502 27227 heartbeater.cc:511] Master 127.22.11.190:39269 requested a full tablet report, sending...
I20260810 06:50:07.128345 26656 ts_manager.cc:194] Registered new tserver with Master: 6f428203307e446da544161e95244757 (127.22.11.130:46551)
I20260810 06:50:07.129462 26656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:50017
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
W20260810 06:50:07.843863 22574 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.22.11.131: real 0.728s user 0.004s sys 0.010s
I20260810 06:50:07.844585 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39269
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:50:08.134546 27227 heartbeater.cc:503] Master 127.22.11.190:39269 was elected leader, sending a full tablet report...
W20260810 06:50:08.166038 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:08.166319 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:08.166455 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:08.173862 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:08.174006 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:08.174187 27237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:08.183663 27237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:08.184908 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:08.186216 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:08.198781 27248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:08.198813 27246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:08.198870 27244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:08.199409 27237 server_base.cc:1034] running on GCE node
I20260810 06:50:08.199959 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:08.200871 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:08.202059 27237 hybrid_clock.cc:648] HybridClock initialized: now 1786344608202018 us; error 38 us; skew 500 ppm
I20260810 06:50:08.202384 27237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:08 dist-test-slave-f7th krb5kdc[26579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344608, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:08.205403 27237 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:50:08.206717 27237 webserver.cc:460] Webserver started at http://127.22.11.131:39137/ using document root <none> and password file <none>
I20260810 06:50:08.207368 27237 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:08.207479 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:08.207770 27237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:08.210636 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "c451bd9fd5ab4f04830df5eada972648"
format_stamp: "Formatted at 2026-08-10 06:50:08 on dist-test-slave-f7th"
server_key: "81b15c0ce3afaa703bfe14913d2122df"
server_key_iv: "961a870619010fe0d72c0fa9e241714a"
server_key_version: "encryptionkey@0"
I20260810 06:50:08.211436 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "c451bd9fd5ab4f04830df5eada972648"
format_stamp: "Formatted at 2026-08-10 06:50:08 on dist-test-slave-f7th"
server_key: "81b15c0ce3afaa703bfe14913d2122df"
server_key_iv: "961a870619010fe0d72c0fa9e241714a"
server_key_version: "encryptionkey@0"
I20260810 06:50:08.216324 27237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:50:08.219556 27259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:08.220834 27237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:50:08.220973 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "c451bd9fd5ab4f04830df5eada972648"
format_stamp: "Formatted at 2026-08-10 06:50:08 on dist-test-slave-f7th"
server_key: "81b15c0ce3afaa703bfe14913d2122df"
server_key_iv: "961a870619010fe0d72c0fa9e241714a"
server_key_version: "encryptionkey@0"
I20260810 06:50:08.221154 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:08.240221 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:08.243856 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:08.244107 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:08.245026 27237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:08.246411 27237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:08.246488 27237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:08.246614 27237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:08.246667 27237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:08.276482 27237 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:44283
I20260810 06:50:08.276527 27462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:44283 every 8 connection(s)
I20260810 06:50:08.277711 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:08.283684 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27237
I20260810 06:50:08.283886 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:08.284432 22574 external_mini_cluster.cc:1428] Setting key ab9b7626c985805a11d43ebb170b08f5
Aug 10 06:50:08 dist-test-slave-f7th krb5kdc[26579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344608, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:08.296372 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:08.280187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:57147 (local address 127.22.11.190:39269)
0810 06:50:08.280693 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:08.280698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:08.281574 (+ 876us) server_negotiation.cc:407] Connection header received
0810 06:50:08.283295 (+ 1721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:08.283299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:08.283383 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:08.283476 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:08.285404 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:08.286322 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:08.288057 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:08.288441 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:08.288552 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:08.291645 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:08.291650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:08.291653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:08.291686 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:08.294035 (+ 2349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:08.294640 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:08.294644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:08.294646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:08.294752 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:08.295058 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:08.295061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:08.295063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:08.295301 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:08.295405 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:08.295947 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 06:50:08.296098 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":235,"threads_started":1}
I20260810 06:50:08.297223 27472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:08.280754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39269 (local address 127.22.11.131:57147)
0810 06:50:08.281419 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:08.281461 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:08.283045 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:08.283766 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:08.283778 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:08.284485 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:08.285217 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:08.285242 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:08.286476 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:08.286481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:08.287931 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:08.287944 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:08.288665 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:08.288673 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:08.288860 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:08.289445 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:08.289472 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:08.291462 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:08.294142 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:08.294148 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:08.294159 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:08.294554 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:08.294820 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:08.294823 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:08.294825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:08.294983 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:08.295385 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:08.295390 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:08.295666 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 06:50:08.295866 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 06:50:08.296198 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":248,"threads_started":1}
I20260810 06:50:08.298565 27467 heartbeater.cc:347] Connected to a master server at 127.22.11.190:39269
I20260810 06:50:08.298902 27467 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:08.299742 27467 heartbeater.cc:511] Master 127.22.11.190:39269 requested a full tablet report, sending...
I20260810 06:50:08.301100 26656 ts_manager.cc:194] Registered new tserver with Master: c451bd9fd5ab4f04830df5eada972648 (127.22.11.131:44283)
I20260810 06:50:08.301831 26656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:57147
I20260810 06:50:08.311337 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:50:08.344513 27469 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:08.342232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43762 (local address 127.22.11.190:39269)
0810 06:50:08.342399 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:08.342404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:08.342856 (+ 452us) server_negotiation.cc:407] Connection header received
0810 06:50:08.343035 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:08.343039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:08.343137 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:08.343217 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:08.344315 (+ 1098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43762: BlockingRecv error: recv got EOF from 127.0.0.1:43762 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:50:08.348330 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26759
W20260810 06:50:08.350982 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26759: Not authorized: Client connection negotiation failed: client connection to 127.22.11.129:39569: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:50:08.351150 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26759
I20260810 06:50:08.372970 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26997
W20260810 06:50:08.375670 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26997: Not authorized: Client connection negotiation failed: client connection to 127.22.11.130:46551: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:50:08.375810 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26997
I20260810 06:50:08.411077 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27237
W20260810 06:50:08.413745 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27237: Not authorized: Client connection negotiation failed: client connection to 127.22.11.131:44283: server requires authentication, but client does not have Kerberos credentials available
I20260810 06:50:08.413864 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27237
I20260810 06:50:08.438877 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26600
I20260810 06:50:08.440104 26710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:08.543269 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26600
2026-08-10T06:50:08Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4990 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:08 dist-test-slave-f7th krb5kdc[27497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:08 dist-test-slave-f7th krb5kdc[27497](info): set up 2 sockets
Aug 10 06:50:08 dist-test-slave-f7th krb5kdc[27497](info): commencing operation
krb5kdc: starting...
W20260810 06:50:09.774523 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:09 dist-test-slave-f7th krb5kdc[27497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:09.968835 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:46135
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:41475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:46135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:10.282158 27527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:10.282434 27527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:10.282572 27527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:10.289647 27527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:10.289750 27527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:10.289834 27527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:10.289913 27527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:10.289980 27527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:10.299702 27527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:41475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:46135
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:46135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:10.300968 27527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:10.302214 27527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:10.313242 27536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:10.313215 27539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:10.313292 27537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:10.314400 27527 server_base.cc:1034] running on GCE node
I20260810 06:50:10.315163 27527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:10.316403 27527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:10.317607 27527 hybrid_clock.cc:648] HybridClock initialized: now 1786344610317554 us; error 37 us; skew 500 ppm
I20260810 06:50:10.317934 27527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:10 dist-test-slave-f7th krb5kdc[27497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344610, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:10.320829 27527 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:10.322192 27527 webserver.cc:460] Webserver started at http://127.22.11.190:36381/ using document root <none> and password file <none>
I20260810 06:50:10.322870 27527 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:10.322989 27527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:10.323271 27527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:10.326119 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "83bd81ae172b4881b53c644d8e016e02"
format_stamp: "Formatted at 2026-08-10 06:50:10 on dist-test-slave-f7th"
server_key: "34541350207fde4782aa5b7285fa9bb3"
server_key_iv: "ea502f90e590aff30374dd7dabc3c49b"
server_key_version: "encryptionkey@0"
I20260810 06:50:10.326870 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "83bd81ae172b4881b53c644d8e016e02"
format_stamp: "Formatted at 2026-08-10 06:50:10 on dist-test-slave-f7th"
server_key: "34541350207fde4782aa5b7285fa9bb3"
server_key_iv: "ea502f90e590aff30374dd7dabc3c49b"
server_key_version: "encryptionkey@0"
I20260810 06:50:10.331571 27527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 06:50:10.334864 27549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:10.336057 27527 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:50:10.336220 27527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "83bd81ae172b4881b53c644d8e016e02"
format_stamp: "Formatted at 2026-08-10 06:50:10 on dist-test-slave-f7th"
server_key: "34541350207fde4782aa5b7285fa9bb3"
server_key_iv: "ea502f90e590aff30374dd7dabc3c49b"
server_key_version: "encryptionkey@0"
I20260810 06:50:10.336391 27527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:10.368191 27527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:10.371737 27527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:10.371990 27527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:10.388861 27527 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:46135
I20260810 06:50:10.388929 27652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:46135 every 8 connection(s)
I20260810 06:50:10.390129 27527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:10.393790 27655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:10.393883 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27527
I20260810 06:50:10.394104 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:10.394743 22574 external_mini_cluster.cc:1428] Setting key 1e7e397a0a55f46da8807158afd0b199
I20260810 06:50:10.407763 27655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02: Bootstrap starting.
Aug 10 06:50:10 dist-test-slave-f7th krb5kdc[27497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:10.410825 27655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:10.412202 27655 log.cc:824] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:10.415378 27655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02: No bootstrap required, opened a new log
I20260810 06:50:10.418763 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:10.396469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34692 (local address 127.22.11.190:46135)
0810 06:50:10.397368 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:10.397383 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:10.397440 (+ 57us) server_negotiation.cc:407] Connection header received
0810 06:50:10.400166 (+ 2726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:10.400194 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:10.400716 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:10.401161 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:10.402432 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:10.404377 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:10.405642 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:10.406699 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:10.406895 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:10.410680 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:10.410691 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:10.410714 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:10.410763 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:10.414464 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:10.415085 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:10.415093 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:10.415101 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:10.415303 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:10.415659 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:10.415664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:10.415667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:10.416379 (+ 712us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:10.416706 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:10.417017 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 06:50:10.417436 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":346,"threads_started":1}
I20260810 06:50:10.422590 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:50:10.425169 27655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83bd81ae172b4881b53c644d8e016e02" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46135 } }
I20260810 06:50:10.425578 27655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:10.425673 27655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83bd81ae172b4881b53c644d8e016e02, State: Initialized, Role: FOLLOWER
I20260810 06:50:10.426290 27655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83bd81ae172b4881b53c644d8e016e02" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46135 } }
I20260810 06:50:10.426561 27655 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:10.426689 27655 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:10.426851 27655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:10.429664 27655 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83bd81ae172b4881b53c644d8e016e02" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46135 } }
I20260810 06:50:10.430194 27655 leader_election.cc:302] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83bd81ae172b4881b53c644d8e016e02; no voters:
I20260810 06:50:10.430796 27655 leader_election.cc:288] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:10.430886 27661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:10.431985 27661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [term 1 LEADER]: Becoming Leader. State: Replica: 83bd81ae172b4881b53c644d8e016e02, State: Running, Role: LEADER
I20260810 06:50:10.432475 27661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83bd81ae172b4881b53c644d8e016e02" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46135 } }
I20260810 06:50:10.432878 27655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:10.435374 27663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83bd81ae172b4881b53c644d8e016e02. Latest consensus state: current_term: 1 leader_uuid: "83bd81ae172b4881b53c644d8e016e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83bd81ae172b4881b53c644d8e016e02" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46135 } } }
I20260810 06:50:10.435729 27663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:10.436265 27662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83bd81ae172b4881b53c644d8e016e02" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83bd81ae172b4881b53c644d8e016e02" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46135 } } }
I20260810 06:50:10.436540 27662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:10.446780 27672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:10.452056 27672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:10.474830 27672 catalog_manager.cc:1347] Generated new cluster ID: 5abee8da6f5f43699f0322d141ea9f32
I20260810 06:50:10.474956 27672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:10.487610 27672 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:10.489697 27672 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:10.503355 27672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83bd81ae172b4881b53c644d8e016e02: Generated new TSK 0
I20260810 06:50:10.504508 27672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:50:10.513986 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:10.495671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.22.11.190:46135)
0810 06:50:10.495840 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:10.495847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:10.505244 (+ 9397us) server_negotiation.cc:407] Connection header received
0810 06:50:10.505723 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:10.505728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:10.505824 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:10.505898 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:10.508371 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:10.509151 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:10.510974 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:10.511426 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:10.511542 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:10.512638 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:10.512645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:10.512650 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:10.513325 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:10.513386 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:10.513409 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:10.513491 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:10.513637 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 06:50:10.513714 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260810 06:50:10.520375 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27527
I20260810 06:50:10.521575 27646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:10.611187 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27527
2026-08-10T06:50:10Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2060 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:10 dist-test-slave-f7th krb5kdc[27718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:10 dist-test-slave-f7th krb5kdc[27718](info): set up 2 sockets
Aug 10 06:50:10 dist-test-slave-f7th krb5kdc[27718](info): commencing operation
krb5kdc: starting...
W20260810 06:50:11.822302 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:11 dist-test-slave-f7th krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:12.011263 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:44041
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:12.327715 27740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:12.327996 27740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:12.328125 27740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:12.335111 27740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:12.335227 27740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:12.335304 27740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:12.335381 27740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:12.335438 27740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:12.344987 27740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:44041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:12.346179 27740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:12.347399 27740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:12.358408 27749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:12.358381 27750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:12.358341 27752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:12.359254 27740 server_base.cc:1034] running on GCE node
I20260810 06:50:12.360016 27740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:12.361335 27740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:12.362547 27740 hybrid_clock.cc:648] HybridClock initialized: now 1786344612362485 us; error 32 us; skew 500 ppm
I20260810 06:50:12.362874 27740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:12 dist-test-slave-f7th krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344612, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:12.368959 27740 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:12.370302 27740 webserver.cc:460] Webserver started at http://127.22.11.190:46597/ using document root <none> and password file <none>
I20260810 06:50:12.370932 27740 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:12.371049 27740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:12.371325 27740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:12.374181 27740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "7bc118698e694b3f8f274edda6039fee"
format_stamp: "Formatted at 2026-08-10 06:50:12 on dist-test-slave-f7th"
server_key: "711a0441e8bc88789a44902f2eb14dd2"
server_key_iv: "3afb30568c3b7f2784117259b2753244"
server_key_version: "encryptionkey@0"
I20260810 06:50:12.374912 27740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "7bc118698e694b3f8f274edda6039fee"
format_stamp: "Formatted at 2026-08-10 06:50:12 on dist-test-slave-f7th"
server_key: "711a0441e8bc88789a44902f2eb14dd2"
server_key_iv: "3afb30568c3b7f2784117259b2753244"
server_key_version: "encryptionkey@0"
I20260810 06:50:12.379698 27740 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:50:12.382929 27763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:12.384137 27740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:50:12.384294 27740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "7bc118698e694b3f8f274edda6039fee"
format_stamp: "Formatted at 2026-08-10 06:50:12 on dist-test-slave-f7th"
server_key: "711a0441e8bc88789a44902f2eb14dd2"
server_key_iv: "3afb30568c3b7f2784117259b2753244"
server_key_version: "encryptionkey@0"
I20260810 06:50:12.384455 27740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:12.397418 27740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:12.401029 27740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:12.401268 27740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:12.417977 27740 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:44041
I20260810 06:50:12.418018 27850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:44041 every 8 connection(s)
I20260810 06:50:12.419261 27740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:12.422755 27851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:12.426741 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27740
I20260810 06:50:12.426975 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:12.427600 22574 external_mini_cluster.cc:1428] Setting key 5b302e6bc296a252b06eba05049b67f8
I20260810 06:50:12.433223 27851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee: Bootstrap starting.
I20260810 06:50:12.436594 27851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:12.437775 27851 log.cc:824] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:12.440939 27851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee: No bootstrap required, opened a new log
Aug 10 06:50:12 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:12.449792 27857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:12.429427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.22.11.190:44041)
0810 06:50:12.430198 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:12.430212 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:12.430255 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:50:12.431972 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:12.431993 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:12.432348 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:12.432692 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:12.434374 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:12.435659 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:12.437405 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:12.438080 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:12.438240 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:12.442805 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:12.442812 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:12.442825 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:12.442866 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:12.445972 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:12.446707 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:12.446716 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:12.446721 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:12.446894 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:12.447272 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:12.447275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:12.447277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:12.447800 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:12.448037 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:12.448448 (+ 411us) server_negotiation.cc:299] Negotiation successful
0810 06:50:12.448854 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":366,"threads_started":1}
I20260810 06:50:12.449637 27851 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc118698e694b3f8f274edda6039fee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44041 } }
I20260810 06:50:12.449985 27851 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:12.450134 27851 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bc118698e694b3f8f274edda6039fee, State: Initialized, Role: FOLLOWER
I20260810 06:50:12.450778 27851 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc118698e694b3f8f274edda6039fee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44041 } }
I20260810 06:50:12.451036 27851 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:12.451179 27851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:12.451376 27851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:12.453913 27851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc118698e694b3f8f274edda6039fee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44041 } }
I20260810 06:50:12.454392 27851 leader_election.cc:302] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bc118698e694b3f8f274edda6039fee; no voters:
I20260810 06:50:12.454931 27851 leader_election.cc:288] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:12.455070 27859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:12.455624 27859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [term 1 LEADER]: Becoming Leader. State: Replica: 7bc118698e694b3f8f274edda6039fee, State: Running, Role: LEADER
I20260810 06:50:12.456233 27859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc118698e694b3f8f274edda6039fee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44041 } }
I20260810 06:50:12.457635 27851 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:12.458865 27862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bc118698e694b3f8f274edda6039fee. Latest consensus state: current_term: 1 leader_uuid: "7bc118698e694b3f8f274edda6039fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc118698e694b3f8f274edda6039fee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44041 } } }
I20260810 06:50:12.459208 27862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:12.459787 27864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bc118698e694b3f8f274edda6039fee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bc118698e694b3f8f274edda6039fee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44041 } } }
I20260810 06:50:12.460213 27864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:12.461373 27871 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:12.465512 27871 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:12.478884 27871 catalog_manager.cc:1347] Generated new cluster ID: ba6b82988c0e4135ada94f8b155e5f60
I20260810 06:50:12.478984 27871 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:12.498593 27871 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:12.500248 27871 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:12.517318 27871 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bc118698e694b3f8f274edda6039fee: Generated new TSK 0
I20260810 06:50:12.518299 27871 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:50:12.574478 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44041
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:12.901463 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:12.901737 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:12.901888 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:12.909543 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:12.909662 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:12.909817 27898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:12.919224 27898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:12.920447 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:12.921700 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:12.934221 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:12.934262 27915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:12.934180 27906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:12.935128 27898 server_base.cc:1034] running on GCE node
I20260810 06:50:12.935703 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:12.936560 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:12.937774 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786344612937725 us; error 42 us; skew 500 ppm
I20260810 06:50:12.938066 27898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:12 dist-test-slave-f7th krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344612, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:12.941028 27898 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:50:12.942301 27898 webserver.cc:460] Webserver started at http://127.22.11.129:37559/ using document root <none> and password file <none>
I20260810 06:50:12.942929 27898 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:12.943051 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:12.943325 27898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:12.946063 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "c75c4b04bee04fd0a20f8279ccd4e916"
format_stamp: "Formatted at 2026-08-10 06:50:12 on dist-test-slave-f7th"
server_key: "81b193adeebbaf7b514b6cd3c436cd9d"
server_key_iv: "dfa43537b5f6b8b56f699b98f54f197f"
server_key_version: "encryptionkey@0"
I20260810 06:50:12.946794 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "c75c4b04bee04fd0a20f8279ccd4e916"
format_stamp: "Formatted at 2026-08-10 06:50:12 on dist-test-slave-f7th"
server_key: "81b193adeebbaf7b514b6cd3c436cd9d"
server_key_iv: "dfa43537b5f6b8b56f699b98f54f197f"
server_key_version: "encryptionkey@0"
I20260810 06:50:12.951719 27898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:50:12.955154 27930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:12.956364 27898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:50:12.956511 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "c75c4b04bee04fd0a20f8279ccd4e916"
format_stamp: "Formatted at 2026-08-10 06:50:12 on dist-test-slave-f7th"
server_key: "81b193adeebbaf7b514b6cd3c436cd9d"
server_key_iv: "dfa43537b5f6b8b56f699b98f54f197f"
server_key_version: "encryptionkey@0"
I20260810 06:50:12.956736 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:12.983975 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:12.987531 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:12.987797 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:12.988703 27898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:12.989954 27898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:12.990031 27898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:12.990145 27898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:12.990198 27898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:13.020038 27898 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:34711
I20260810 06:50:13.020078 28124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:34711 every 8 connection(s)
I20260810 06:50:13.021347 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:13.023890 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27898
I20260810 06:50:13.024108 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:13.024958 22574 external_mini_cluster.cc:1428] Setting key ab9bb987c49185517b6146f9ee1ce7b7
Aug 10 06:50:13 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344612, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:13.041350 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:13.023809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:34087 (local address 127.22.11.190:44041)
0810 06:50:13.024229 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:13.024234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:13.025453 (+ 1219us) server_negotiation.cc:407] Connection header received
0810 06:50:13.027383 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:13.027389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:13.027469 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:13.027568 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:13.029057 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.029795 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.031315 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.031704 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.031824 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:13.035507 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:13.035514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:13.035519 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:13.035556 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:13.038538 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:13.039199 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:13.039204 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:13.039208 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:13.039367 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:13.039740 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:13.039744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:13.039746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:13.040046 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:13.040179 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:13.040801 (+ 622us) server_negotiation.cc:299] Negotiation successful
0810 06:50:13.041021 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":212,"threads_started":1}
I20260810 06:50:13.042006 28134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:13.024478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:44041 (local address 127.22.11.129:34087)
0810 06:50:13.025295 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:13.025338 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:13.027132 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:13.027802 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:13.027810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:13.028307 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:13.028898 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:13.028916 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.029940 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.029944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:13.031170 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:13.031183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.031925 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.031933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:13.032160 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:13.032798 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:13.032820 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:13.035283 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:13.038657 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:13.038663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:13.038673 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:13.039065 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:13.039455 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:13.039458 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:13.039460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:13.039633 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:13.040140 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:13.040145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:13.040422 (+ 277us) client_negotiation.cc:769] Sending connection context
0810 06:50:13.040648 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 06:50:13.041012 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":298,"threads_started":1}
I20260810 06:50:13.043483 28125 heartbeater.cc:347] Connected to a master server at 127.22.11.190:44041
I20260810 06:50:13.043821 28125 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:13.044750 28125 heartbeater.cc:511] Master 127.22.11.190:44041 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
I20260810 06:50:13.046881 27791 ts_manager.cc:194] Registered new tserver with Master: c75c4b04bee04fd0a20f8279ccd4e916 (127.22.11.129:34711)
I20260810 06:50:13.048384 27791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:34087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:50:13.071705 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44041
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:13.386766 28138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:13.387037 28138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:13.387138 28138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:13.394541 28138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:13.394635 28138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:13.394750 28138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:13.403801 28138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:13.404953 28138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:13.406243 28138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:13.418476 28152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:13.418514 28150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:13.424588 28138 server_base.cc:1034] running on GCE node
W20260810 06:50:13.425294 28146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:13.426029 28138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:13.427106 28138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:13.428333 28138 hybrid_clock.cc:648] HybridClock initialized: now 1786344613428249 us; error 74 us; skew 500 ppm
I20260810 06:50:13.428725 28138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:13 dist-test-slave-f7th krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:13.432397 28138 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:50:13.434054 28138 webserver.cc:460] Webserver started at http://127.22.11.130:33203/ using document root <none> and password file <none>
I20260810 06:50:13.434811 28138 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:13.434914 28138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:13.435169 28138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:13.437922 28138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "2f07c6b9237945cb864808467ea2ec61"
format_stamp: "Formatted at 2026-08-10 06:50:13 on dist-test-slave-f7th"
server_key: "0548263e9b237a7f3c740bf0dfa1438b"
server_key_iv: "1d93513af06fa9db168f44b7b4e5fe56"
server_key_version: "encryptionkey@0"
I20260810 06:50:13.438606 28138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "2f07c6b9237945cb864808467ea2ec61"
format_stamp: "Formatted at 2026-08-10 06:50:13 on dist-test-slave-f7th"
server_key: "0548263e9b237a7f3c740bf0dfa1438b"
server_key_iv: "1d93513af06fa9db168f44b7b4e5fe56"
server_key_version: "encryptionkey@0"
I20260810 06:50:13.443281 28138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 06:50:13.446452 28161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:13.447665 28138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:50:13.447788 28138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "2f07c6b9237945cb864808467ea2ec61"
format_stamp: "Formatted at 2026-08-10 06:50:13 on dist-test-slave-f7th"
server_key: "0548263e9b237a7f3c740bf0dfa1438b"
server_key_iv: "1d93513af06fa9db168f44b7b4e5fe56"
server_key_version: "encryptionkey@0"
I20260810 06:50:13.447947 28138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:13.475409 28138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:13.479065 28138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:13.479315 28138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:13.480190 28138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:13.481518 28138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:13.481587 28138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:13.481678 28138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:13.481719 28138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:13.513155 28138 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:32971
I20260810 06:50:13.513198 28366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:32971 every 8 connection(s)
I20260810 06:50:13.514392 28138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:13.518200 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28138
I20260810 06:50:13.518388 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:13.518958 22574 external_mini_cluster.cc:1428] Setting key 2f620c14b1095055165e21daf58b69a1
Aug 10 06:50:13 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:13.534881 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:13.516773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:37661 (local address 127.22.11.190:44041)
0810 06:50:13.516973 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:13.516979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:13.518362 (+ 1383us) server_negotiation.cc:407] Connection header received
0810 06:50:13.520592 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:13.520597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:13.520700 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:13.520810 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:13.522666 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.523397 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.525412 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.525826 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.525918 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:13.529646 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:13.529651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:13.529654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:13.529686 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:13.531998 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:13.532698 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:13.532702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:13.532704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:13.532835 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:13.533250 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:13.533253 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:13.533255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:13.533519 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:13.533650 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:13.534309 (+ 659us) server_negotiation.cc:299] Negotiation successful
0810 06:50:13.534555 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:50:13.535745 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:13.517358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:44041 (local address 127.22.11.130:37661)
0810 06:50:13.518199 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:13.518254 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:13.520336 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:13.521048 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:13.521059 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:13.521764 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:13.522502 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:13.522527 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.523551 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.523556 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:13.525291 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:13.525305 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:13.526031 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:13.526046 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:13.526295 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:13.526950 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:13.526977 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:13.529412 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:13.532125 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:13.532134 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:13.532149 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:13.532552 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:13.532938 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:13.532943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:13.532947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:13.533134 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:13.533641 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:13.533649 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:13.533949 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 06:50:13.534181 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 06:50:13.534659 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":272,"threads_started":1}
I20260810 06:50:13.537076 28368 heartbeater.cc:347] Connected to a master server at 127.22.11.190:44041
I20260810 06:50:13.537413 28368 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:13.538286 28368 heartbeater.cc:511] Master 127.22.11.190:44041 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
I20260810 06:50:13.539808 27791 ts_manager.cc:194] Registered new tserver with Master: 2f07c6b9237945cb864808467ea2ec61 (127.22.11.130:32971)
I20260810 06:50:13.540571 27791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:37661
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:50:13.563390 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44041
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:13.881160 28380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:13.881407 28380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:13.881537 28380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:13.888553 28380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:13.888681 28380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:13.888808 28380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:13.898065 28380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:13.899194 28380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:13.900424 28380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:13.912775 28388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:13.912863 28390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:13.912990 28386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:13.913621 28380 server_base.cc:1034] running on GCE node
I20260810 06:50:13.914208 28380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:13.915011 28380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:13.916210 28380 hybrid_clock.cc:648] HybridClock initialized: now 1786344613916156 us; error 42 us; skew 500 ppm
I20260810 06:50:13.916495 28380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:13 dist-test-slave-f7th krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:13.919457 28380 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:50:13.920737 28380 webserver.cc:460] Webserver started at http://127.22.11.131:42337/ using document root <none> and password file <none>
I20260810 06:50:13.921343 28380 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:13.921442 28380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:13.921677 28380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:13.924332 28380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "90837061a3654c42b209eac3e7a70d8a"
format_stamp: "Formatted at 2026-08-10 06:50:13 on dist-test-slave-f7th"
server_key: "ceb4f630af52ee9ba676a7964d955571"
server_key_iv: "f8e91b56b7c259758a60bba035f12bb6"
server_key_version: "encryptionkey@0"
I20260810 06:50:13.925076 28380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "90837061a3654c42b209eac3e7a70d8a"
format_stamp: "Formatted at 2026-08-10 06:50:13 on dist-test-slave-f7th"
server_key: "ceb4f630af52ee9ba676a7964d955571"
server_key_iv: "f8e91b56b7c259758a60bba035f12bb6"
server_key_version: "encryptionkey@0"
I20260810 06:50:13.929731 28380 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:50:13.932969 28404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:13.934152 28380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 06:50:13.934295 28380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "90837061a3654c42b209eac3e7a70d8a"
format_stamp: "Formatted at 2026-08-10 06:50:13 on dist-test-slave-f7th"
server_key: "ceb4f630af52ee9ba676a7964d955571"
server_key_iv: "f8e91b56b7c259758a60bba035f12bb6"
server_key_version: "encryptionkey@0"
I20260810 06:50:13.934456 28380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:13.954824 28380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:13.958856 28380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:13.959110 28380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:13.960060 28380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:13.961405 28380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:13.961480 28380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:13.961593 28380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:13.961642 28380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:13.990898 28380 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:42035
I20260810 06:50:13.990968 28604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:42035 every 8 connection(s)
I20260810 06:50:13.992110 28380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:13.999563 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28380
I20260810 06:50:13.999742 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:14.000269 22574 external_mini_cluster.cc:1428] Setting key e49edc1a8578c4b18c5c8dbc67bf7f5b
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344613, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:14.010571 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:13.994450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:57625 (local address 127.22.11.190:44041)
0810 06:50:13.994626 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:13.994631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:13.995713 (+ 1082us) server_negotiation.cc:407] Connection header received
0810 06:50:13.997520 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:13.997524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:13.997593 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:13.997676 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:13.999012 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.000080 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.001909 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.002410 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.002503 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:14.005823 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:14.005827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:14.005831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:14.005862 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:14.008148 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.008817 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.008821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.008823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.008952 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.009273 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.009276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.009278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.009513 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:14.009627 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:14.010167 (+ 540us) server_negotiation.cc:299] Negotiation successful
0810 06:50:14.010320 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:50:14.011468 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:13.994880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:44041 (local address 127.22.11.131:57625)
0810 06:50:13.995564 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:13.995606 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:13.997289 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:13.997858 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:13.997866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:13.998338 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:13.998880 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:13.998897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.000219 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.000223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:14.001784 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:14.001796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.002636 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.002644 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:14.002920 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:14.003564 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:14.003585 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:14.005637 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:14.008267 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:14.008273 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:14.008283 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:14.008728 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:14.009029 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:14.009032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:14.009034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:14.009197 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:14.009606 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:14.009611 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:14.009882 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 06:50:14.010085 (+ 203us) client_negotiation.cc:240] Negotiation successful
0810 06:50:14.010420 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":259,"threads_started":1}
I20260810 06:50:14.012609 28605 heartbeater.cc:347] Connected to a master server at 127.22.11.190:44041
I20260810 06:50:14.012897 28605 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:14.013638 28605 heartbeater.cc:511] Master 127.22.11.190:44041 requested a full tablet report, sending...
I20260810 06:50:14.015161 27791 ts_manager.cc:194] Registered new tserver with Master: 90837061a3654c42b209eac3e7a70d8a (127.22.11.131:42035)
I20260810 06:50:14.016072 27791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:57625
I20260810 06:50:14.027145 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:50:14.051198 28125 heartbeater.cc:503] Master 127.22.11.190:44041 was elected leader, sending a full tablet report...
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[27718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344614, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344614, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:14.075947 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:14.063135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36336 (local address 127.22.11.190:44041)
0810 06:50:14.063346 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:14.063352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:14.063717 (+ 365us) server_negotiation.cc:407] Connection header received
0810 06:50:14.063850 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:14.063854 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:14.063916 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:14.063992 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:14.065304 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.066023 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.067236 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.067651 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.067752 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:14.071323 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:14.071327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:14.071330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:14.071361 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:14.073598 (+ 2237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.074171 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.074176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.074178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.074310 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.074656 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.074660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.074662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.074960 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:14.075203 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:14.075455 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 06:50:14.075661 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 06:50:14.076476 27791 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36336
I20260810 06:50:14.078199 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27898
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344614, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:14.092718 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:14.079582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.22.11.129:34711)
0810 06:50:14.080135 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:14.080142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:14.080165 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:50:14.080217 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:14.080223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:14.080422 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:14.080544 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:14.081570 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.082521 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.083642 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.084182 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.084274 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:14.087796 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:14.087812 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:14.087829 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:14.087874 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:14.090485 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.091043 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.091050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.091052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.091168 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.091429 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.091432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.091434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.091757 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:14.092024 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:14.092198 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:50:14.092395 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":266,"threads_started":1}
W20260810 06:50:14.093886 28117 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50212
W20260810 06:50:14.094895 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27898: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:50:14.095077 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27898
I20260810 06:50:14.130735 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28138
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344614, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:50:14.145202 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:14.131600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.22.11.130:32971)
0810 06:50:14.132151 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:14.132158 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:14.132182 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:50:14.132240 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:14.132246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:14.132455 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:14.132580 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:14.133721 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.134712 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.135924 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.136453 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.136542 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:14.140441 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:14.140452 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:14.140465 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:14.140496 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:14.142970 (+ 2474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.143526 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.143533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.143535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.143647 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.143902 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.143904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.143906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.144215 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:14.144442 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:14.144652 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 06:50:14.144849 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":281,"threads_started":1}
W20260810 06:50:14.146418 28359 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58260
W20260810 06:50:14.147420 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28138: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:50:14.147571 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28138
I20260810 06:50:14.171154 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28380
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[27718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344614, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:50:14.186168 28634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:14.172073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59580 (local address 127.22.11.131:42035)
0810 06:50:14.172807 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:14.172815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:14.172843 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:50:14.172902 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:14.172908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:14.173111 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:14.173234 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:14.174416 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.175417 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.176518 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:14.177173 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:14.177263 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:14.181155 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:14.181166 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:14.181180 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:14.181217 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:14.183898 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.184437 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.184443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.184445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.184572 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:14.184928 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:14.184931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:14.184933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:14.185257 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:14.185473 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:14.185698 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:50:14.185893 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":379,"threads_started":1}
W20260810 06:50:14.187345 28596 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59580
W20260810 06:50:14.188351 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28380: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:50:14.188547 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28380
I20260810 06:50:14.222051 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27740
I20260810 06:50:14.223119 27841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:14.325542 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27740
2026-08-10T06:50:14Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3721 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[28644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[28644](info): set up 2 sockets
Aug 10 06:50:14 dist-test-slave-f7th krb5kdc[28644](info): commencing operation
krb5kdc: starting...
W20260810 06:50:15.550191 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:15 dist-test-slave-f7th krb5kdc[28644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:15.745966 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37343
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:37343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:16.064953 28665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:16.065248 28665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:16.065384 28665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:16.072445 28665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:16.072557 28665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:16.072654 28665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:16.072734 28665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:16.072782 28665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:16.082156 28665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:37343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:16.083292 28665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:16.084491 28665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:16.095530 28675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:16.095753 28672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:16.095732 28678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:16.096714 28665 server_base.cc:1034] running on GCE node
I20260810 06:50:16.097486 28665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:16.098755 28665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:16.099951 28665 hybrid_clock.cc:648] HybridClock initialized: now 1786344616099905 us; error 32 us; skew 500 ppm
I20260810 06:50:16.100283 28665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:16 dist-test-slave-f7th krb5kdc[28644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344616, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:16.103205 28665 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:16.104563 28665 webserver.cc:460] Webserver started at http://127.22.11.190:36149/ using document root <none> and password file <none>
I20260810 06:50:16.105221 28665 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:16.105329 28665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:16.105618 28665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:16.108439 28665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "0ad36030067e421a8d8c61f236b5eeb7"
format_stamp: "Formatted at 2026-08-10 06:50:16 on dist-test-slave-f7th"
server_key: "1445d42b0871a5093325a8ed7dfe2233"
server_key_iv: "bbf508612537cd1f51de92039682da2c"
server_key_version: "encryptionkey@0"
I20260810 06:50:16.109251 28665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "0ad36030067e421a8d8c61f236b5eeb7"
format_stamp: "Formatted at 2026-08-10 06:50:16 on dist-test-slave-f7th"
server_key: "1445d42b0871a5093325a8ed7dfe2233"
server_key_iv: "bbf508612537cd1f51de92039682da2c"
server_key_version: "encryptionkey@0"
I20260810 06:50:16.114009 28665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:50:16.117187 28687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:16.118374 28665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:16.118536 28665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "0ad36030067e421a8d8c61f236b5eeb7"
format_stamp: "Formatted at 2026-08-10 06:50:16 on dist-test-slave-f7th"
server_key: "1445d42b0871a5093325a8ed7dfe2233"
server_key_iv: "bbf508612537cd1f51de92039682da2c"
server_key_version: "encryptionkey@0"
I20260810 06:50:16.118705 28665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:16.133378 28665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:16.137598 28665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:16.137842 28665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:16.154830 28665 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:37343
I20260810 06:50:16.154877 28772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:37343 every 8 connection(s)
I20260810 06:50:16.156314 28665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:16.160032 28773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:16.161655 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28665
I20260810 06:50:16.161854 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:16.162524 22574 external_mini_cluster.cc:1428] Setting key 3e6ffe01225b8f23190f82c757d40819
I20260810 06:50:16.171654 28773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7: Bootstrap starting.
I20260810 06:50:16.175282 28773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:16.176455 28773 log.cc:824] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:50:16 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:16.179059 28773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7: No bootstrap required, opened a new log
I20260810 06:50:16.184190 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:16.164416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32868 (local address 127.22.11.190:37343)
0810 06:50:16.165213 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:16.165223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:16.165267 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:50:16.166934 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:16.166957 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:16.167310 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:16.167605 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:16.169377 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:16.170661 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:16.172312 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:16.172992 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:16.173143 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:16.177530 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:16.177538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:16.177551 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:16.177589 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:16.180413 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:16.181092 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:16.181099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:16.181104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:16.181279 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:16.181641 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:16.181645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:16.181647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:16.182166 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:16.182453 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:16.182875 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 06:50:16.183250 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":350,"threads_started":1}
I20260810 06:50:16.186733 28773 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad36030067e421a8d8c61f236b5eeb7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37343 } }
I20260810 06:50:16.187170 28773 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:16.187263 28773 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ad36030067e421a8d8c61f236b5eeb7, State: Initialized, Role: FOLLOWER
I20260810 06:50:16.187769 28773 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad36030067e421a8d8c61f236b5eeb7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37343 } }
I20260810 06:50:16.187990 28773 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:16.188120 28773 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:16.188272 28773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:16.190326 28773 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad36030067e421a8d8c61f236b5eeb7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37343 } }
I20260810 06:50:16.190783 28773 leader_election.cc:302] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ad36030067e421a8d8c61f236b5eeb7; no voters:
I20260810 06:50:16.191344 28773 leader_election.cc:288] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:16.191457 28783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:16.192554 28783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ad36030067e421a8d8c61f236b5eeb7, State: Running, Role: LEADER
I20260810 06:50:16.193188 28783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad36030067e421a8d8c61f236b5eeb7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37343 } }
I20260810 06:50:16.193400 28773 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:16.195827 28787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ad36030067e421a8d8c61f236b5eeb7. Latest consensus state: current_term: 1 leader_uuid: "0ad36030067e421a8d8c61f236b5eeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad36030067e421a8d8c61f236b5eeb7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37343 } } }
I20260810 06:50:16.195895 28784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ad36030067e421a8d8c61f236b5eeb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ad36030067e421a8d8c61f236b5eeb7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37343 } } }
I20260810 06:50:16.196280 28787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:16.196287 28784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:16.197952 28794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:16.202996 28794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:16.211511 28794 catalog_manager.cc:1347] Generated new cluster ID: 1f17002cf36344609950589983e67b68
I20260810 06:50:16.211606 28794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:16.228457 28794 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:16.230242 28794 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:16.242483 28794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ad36030067e421a8d8c61f236b5eeb7: Generated new TSK 0
I20260810 06:50:16.243492 28794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:50:16.299688 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37343
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:16.618366 28818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:16.618636 28818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:16.618772 28818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:16.625907 28818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:16.626019 28818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:16.626163 28818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:16.635481 28818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:16.636761 28818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:16.638001 28818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:16.650522 28826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:16.650465 28827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:16.650502 28832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:16.651374 28818 server_base.cc:1034] running on GCE node
I20260810 06:50:16.651970 28818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:16.652884 28818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:16.654102 28818 hybrid_clock.cc:648] HybridClock initialized: now 1786344616654052 us; error 40 us; skew 500 ppm
I20260810 06:50:16.654385 28818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:16 dist-test-slave-f7th krb5kdc[28644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344616, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:16.657362 28818 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:50:16.658751 28818 webserver.cc:460] Webserver started at http://127.22.11.129:36559/ using document root <none> and password file <none>
I20260810 06:50:16.659390 28818 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:16.659509 28818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:16.659797 28818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:16.662650 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "0e8c98c38faa489c8a111fbb90bc34bf"
format_stamp: "Formatted at 2026-08-10 06:50:16 on dist-test-slave-f7th"
server_key: "e2ed9d41e7f954bb5449a3fb767eea90"
server_key_iv: "0b664353d9572dbd0d26cf2e4113a5e0"
server_key_version: "encryptionkey@0"
I20260810 06:50:16.663406 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "0e8c98c38faa489c8a111fbb90bc34bf"
format_stamp: "Formatted at 2026-08-10 06:50:16 on dist-test-slave-f7th"
server_key: "e2ed9d41e7f954bb5449a3fb767eea90"
server_key_iv: "0b664353d9572dbd0d26cf2e4113a5e0"
server_key_version: "encryptionkey@0"
I20260810 06:50:16.668295 28818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:50:16.671582 28844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:16.672992 28818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:50:16.673147 28818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "0e8c98c38faa489c8a111fbb90bc34bf"
format_stamp: "Formatted at 2026-08-10 06:50:16 on dist-test-slave-f7th"
server_key: "e2ed9d41e7f954bb5449a3fb767eea90"
server_key_iv: "0b664353d9572dbd0d26cf2e4113a5e0"
server_key_version: "encryptionkey@0"
I20260810 06:50:16.673332 28818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:16.686607 28818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:16.690248 28818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:16.690505 28818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:16.691385 28818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:16.692739 28818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:16.692814 28818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:16.692914 28818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:16.692960 28818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:16.723446 28818 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:35813
I20260810 06:50:16.723486 29040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:35813 every 8 connection(s)
I20260810 06:50:16.724663 28818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:16.729445 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28818
I20260810 06:50:16.729676 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:16.730485 22574 external_mini_cluster.cc:1428] Setting key c8c7b76bcdd37e917e6389d15c54c0ba
Aug 10 06:50:16 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344616, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:16.745149 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:16.726987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:44901 (local address 127.22.11.190:37343)
0810 06:50:16.727425 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:16.727429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:16.728245 (+ 816us) server_negotiation.cc:407] Connection header received
0810 06:50:16.730626 (+ 2381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:16.730632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:16.730719 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:16.730824 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:16.732674 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:16.733391 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:16.735480 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:16.735871 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:16.735958 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:16.739663 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:16.739668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:16.739671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:16.739702 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:16.742132 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:16.742919 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:16.742923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:16.742926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:16.743052 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:16.743449 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:16.743454 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:16.743457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:16.743720 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:16.743852 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:16.744471 (+ 619us) server_negotiation.cc:299] Negotiation successful
0810 06:50:16.744727 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":221,"threads_started":1}
I20260810 06:50:16.745982 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:16.727396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:37343 (local address 127.22.11.129:44901)
0810 06:50:16.728083 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:16.728123 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:16.730215 (+ 2092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:16.731234 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:16.731243 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:16.731822 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:16.732474 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:16.732500 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:16.733550 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:16.733555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:16.735360 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:16.735375 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:16.736081 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:16.736088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:16.736348 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:16.737032 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:16.737061 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:16.739461 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:16.742277 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:16.742286 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:16.742300 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:16.742810 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:16.743152 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:16.743157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:16.743161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:16.743344 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:16.743843 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:16.743851 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:16.744148 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 06:50:16.744371 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 06:50:16.744893 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":259,"threads_started":1}
I20260810 06:50:16.747406 29041 heartbeater.cc:347] Connected to a master server at 127.22.11.190:37343
I20260810 06:50:16.747682 29041 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:16.748387 29041 heartbeater.cc:511] Master 127.22.11.190:37343 requested a full tablet report, sending...
I20260810 06:50:16.750607 28710 ts_manager.cc:194] Registered new tserver with Master: 0e8c98c38faa489c8a111fbb90bc34bf (127.22.11.129:35813)
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
I20260810 06:50:16.752490 28710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:44901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:50:16.776409 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37343
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:17.095042 29049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:17.095314 29049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:17.095463 29049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:17.102533 29049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:17.102632 29049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:17.102790 29049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:17.112142 29049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:17.113476 29049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:17.114750 29049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:17.127516 29059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:17.127444 29064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:17.127476 29062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:17.128310 29049 server_base.cc:1034] running on GCE node
I20260810 06:50:17.128950 29049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:17.129798 29049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:17.131017 29049 hybrid_clock.cc:648] HybridClock initialized: now 1786344617130965 us; error 30 us; skew 500 ppm
I20260810 06:50:17.131335 29049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:17.134428 29049 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:50:17.135797 29049 webserver.cc:460] Webserver started at http://127.22.11.130:32887/ using document root <none> and password file <none>
I20260810 06:50:17.136485 29049 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:17.136607 29049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:17.136956 29049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:17.139724 29049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "1f40798c497f48858ba7f7461008462a"
format_stamp: "Formatted at 2026-08-10 06:50:17 on dist-test-slave-f7th"
server_key: "44f03e4174e0bc06b8d3cbbdfb7c19e9"
server_key_iv: "5ad16dceb832578ea6fccab3753435c5"
server_key_version: "encryptionkey@0"
I20260810 06:50:17.140496 29049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "1f40798c497f48858ba7f7461008462a"
format_stamp: "Formatted at 2026-08-10 06:50:17 on dist-test-slave-f7th"
server_key: "44f03e4174e0bc06b8d3cbbdfb7c19e9"
server_key_iv: "5ad16dceb832578ea6fccab3753435c5"
server_key_version: "encryptionkey@0"
I20260810 06:50:17.145366 29049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:50:17.148663 29078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:17.149887 29049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:50:17.150045 29049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "1f40798c497f48858ba7f7461008462a"
format_stamp: "Formatted at 2026-08-10 06:50:17 on dist-test-slave-f7th"
server_key: "44f03e4174e0bc06b8d3cbbdfb7c19e9"
server_key_iv: "5ad16dceb832578ea6fccab3753435c5"
server_key_version: "encryptionkey@0"
I20260810 06:50:17.150223 29049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:17.178076 29049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:17.181757 29049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:17.182018 29049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:17.182905 29049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:17.184185 29049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:17.184262 29049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:17.184379 29049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:17.184433 29049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:17.214731 29049 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:39897
I20260810 06:50:17.214769 29271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:39897 every 8 connection(s)
I20260810 06:50:17.215966 29049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:17.222734 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29049
I20260810 06:50:17.222925 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:17.223487 22574 external_mini_cluster.cc:1428] Setting key 6eda146b5eca962c92f9e197d15633c3
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:17.235625 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.218423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:60689 (local address 127.22.11.190:37343)
0810 06:50:17.218596 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:17.218600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:17.219655 (+ 1055us) server_negotiation.cc:407] Connection header received
0810 06:50:17.221464 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:17.221469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:17.221538 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:17.221631 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:17.223400 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.224150 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.225978 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.226376 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.226467 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.230212 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:17.230216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:17.230220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:17.230252 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:17.232730 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.233500 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.233505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.233509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.233681 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.234055 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.234059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.234062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.234361 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:17.234507 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:17.235085 (+ 578us) server_negotiation.cc:299] Negotiation successful
0810 06:50:17.235296 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:50:17.236320 29279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.218854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:37343 (local address 127.22.11.130:60689)
0810 06:50:17.219497 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:17.219540 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:17.221215 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:17.221839 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:17.221848 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:17.222451 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:17.223242 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:17.223267 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.224293 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.224298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:17.225837 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:17.225857 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.226581 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.226589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:17.226874 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.227527 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:17.227557 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:17.230003 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:17.232862 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:17.232869 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:17.232883 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:17.233392 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:17.233778 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:17.233782 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:17.233784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:17.233953 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:17.234462 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:17.234467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:17.234734 (+ 267us) client_negotiation.cc:769] Sending connection context
0810 06:50:17.234938 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 06:50:17.235296 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":253,"threads_started":1}
I20260810 06:50:17.237716 29273 heartbeater.cc:347] Connected to a master server at 127.22.11.190:37343
I20260810 06:50:17.237990 29273 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:17.238663 29273 heartbeater.cc:511] Master 127.22.11.190:37343 requested a full tablet report, sending...
I20260810 06:50:17.240090 28710 ts_manager.cc:194] Registered new tserver with Master: 1f40798c497f48858ba7f7461008462a (127.22.11.130:39897)
I20260810 06:50:17.240927 28710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:60689
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:50:17.269737 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37343
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:17.584149 29284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:17.584407 29284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:17.584530 29284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:17.591557 29284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:17.591667 29284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:17.591821 29284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:17.601322 29284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:17.602492 29284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:17.603745 29284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:17.615861 29297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:17.616010 29292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:17.616043 29295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:17.616902 29284 server_base.cc:1034] running on GCE node
I20260810 06:50:17.617463 29284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:17.618333 29284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:17.619526 29284 hybrid_clock.cc:648] HybridClock initialized: now 1786344617619481 us; error 38 us; skew 500 ppm
I20260810 06:50:17.619809 29284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:17.625392 29284 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:50:17.626737 29284 webserver.cc:460] Webserver started at http://127.22.11.131:43421/ using document root <none> and password file <none>
I20260810 06:50:17.627372 29284 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:17.627487 29284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:17.627777 29284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:17.630582 29284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "1130dfd85cff433f9c503c3b37bddabd"
format_stamp: "Formatted at 2026-08-10 06:50:17 on dist-test-slave-f7th"
server_key: "5ffec25f4eadbbe650a01d370014d8f6"
server_key_iv: "359580334d75fbab8a770ffd1bde5b65"
server_key_version: "encryptionkey@0"
I20260810 06:50:17.631315 29284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "1130dfd85cff433f9c503c3b37bddabd"
format_stamp: "Formatted at 2026-08-10 06:50:17 on dist-test-slave-f7th"
server_key: "5ffec25f4eadbbe650a01d370014d8f6"
server_key_iv: "359580334d75fbab8a770ffd1bde5b65"
server_key_version: "encryptionkey@0"
I20260810 06:50:17.636103 29284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:50:17.639360 29306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:17.640527 29284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:50:17.640714 29284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "1130dfd85cff433f9c503c3b37bddabd"
format_stamp: "Formatted at 2026-08-10 06:50:17 on dist-test-slave-f7th"
server_key: "5ffec25f4eadbbe650a01d370014d8f6"
server_key_iv: "359580334d75fbab8a770ffd1bde5b65"
server_key_version: "encryptionkey@0"
I20260810 06:50:17.640890 29284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:17.658834 29284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:17.662448 29284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:17.662696 29284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:17.663574 29284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:17.664870 29284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:17.664940 29284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:17.665056 29284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:17.665108 29284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:17.695389 29284 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:39289
I20260810 06:50:17.695451 29496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:39289 every 8 connection(s)
I20260810 06:50:17.696635 29284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:17.706235 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29284
I20260810 06:50:17.706418 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:17.706846 22574 external_mini_cluster.cc:1428] Setting key 75d4e875648791cc7a8a371d2a3ef2dc
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:17.714444 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.699040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:45759 (local address 127.22.11.190:37343)
0810 06:50:17.699200 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:17.699204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:17.700327 (+ 1123us) server_negotiation.cc:407] Connection header received
0810 06:50:17.702122 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:17.702128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:17.702211 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:17.702299 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:17.703716 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.704450 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.705966 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.706395 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.706517 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.709825 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:17.709830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:17.709833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:17.709867 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:17.712080 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.712711 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.712714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.712716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.712832 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.713159 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.713161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.713163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.713404 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:17.713508 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:17.714050 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 06:50:17.714205 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:50:17.715308 29501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.699473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:37343 (local address 127.22.11.131:45759)
0810 06:50:17.700172 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:17.700208 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:17.701862 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:17.702534 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:17.702542 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:17.703020 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:17.703554 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:17.703570 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.704601 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.704605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:17.705822 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:17.705835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.706606 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.706615 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:17.706854 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.707455 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:17.707477 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:17.709643 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:17.712189 (+ 2546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:17.712195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:17.712206 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:17.712597 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:17.712907 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:17.712910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:17.712912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:17.713081 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:17.713487 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:17.713505 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:17.713776 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 06:50:17.713981 (+ 205us) client_negotiation.cc:240] Negotiation successful
0810 06:50:17.714310 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":264,"threads_started":1}
I20260810 06:50:17.716451 29497 heartbeater.cc:347] Connected to a master server at 127.22.11.190:37343
I20260810 06:50:17.716732 29497 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:17.717394 29497 heartbeater.cc:511] Master 127.22.11.190:37343 requested a full tablet report, sending...
I20260810 06:50:17.718700 28710 ts_manager.cc:194] Registered new tserver with Master: 1130dfd85cff433f9c503c3b37bddabd (127.22.11.131:39289)
I20260810 06:50:17.719453 28710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:45759
I20260810 06:50:17.721757 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260810 06:50:17.755594 29041 heartbeater.cc:503] Master 127.22.11.190:37343 was elected leader, sending a full tablet report...
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:17.771242 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.757030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46298 (local address 127.22.11.129:35813)
0810 06:50:17.757848 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:17.757855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:17.757877 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:50:17.757927 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:17.757932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:17.758124 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:17.758235 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:17.759517 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.760517 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.761723 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.762257 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.762349 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.765964 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:17.765975 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:17.765989 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:17.766024 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:17.768590 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.769231 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.769238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.769240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.769372 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.769741 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.769745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.769747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.770076 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:17.770357 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:17.770649 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 06:50:17.770908 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":535,"threads_started":1}
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:17.788319 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.775374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32876 (local address 127.22.11.190:37343)
0810 06:50:17.775571 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:17.775577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:17.775728 (+ 151us) server_negotiation.cc:407] Connection header received
0810 06:50:17.775879 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:17.775883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:17.775943 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:17.776024 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:17.777289 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.778018 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.779176 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.779596 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.779703 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.783157 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:17.783161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:17.783164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:17.783195 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:17.785450 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.786023 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.786027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.786030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.786197 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.786603 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.786609 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.786612 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.787038 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:17.787255 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:17.787814 (+ 559us) server_negotiation.cc:299] Negotiation successful
0810 06:50:17.788018 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260810 06:50:17.788913 28710 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32876
I20260810 06:50:17.789916 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28818
I20260810 06:50:17.804533 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.792899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.22.11.129:35813)
0810 06:50:17.793061 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:17.793066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:17.793081 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:50:17.793129 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:17.793133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:17.793195 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:17.793269 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:17.794655 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.795481 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.796685 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.797107 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.797219 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.799687 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:17.799692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:17.799696 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:17.799727 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:17.802144 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.802765 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.802771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.802775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.802898 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.803232 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.803238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.803241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.803492 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:17.803712 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:17.804012 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 06:50:17.804237 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:50:17.805102 29034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:17.887166 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28818
I20260810 06:50:17.912834 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29049
Aug 10 06:50:17 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:50:17.927888 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:17.913741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.22.11.130:39897)
0810 06:50:17.914232 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:17.914239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:17.914264 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:50:17.914324 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:17.914330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:17.914540 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:17.914664 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:17.915832 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.916854 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.917986 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:17.918507 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:17.918598 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:17.922375 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:17.922386 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:17.922399 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:17.922437 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:17.925247 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.925878 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.925885 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.925887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.926018 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:17.926358 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:17.926361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:17.926363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:17.926704 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:17.926992 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:17.927302 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 06:50:17.927568 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":241,"threads_started":1}
I20260810 06:50:17.929072 29264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:18.009755 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29049
I20260810 06:50:18.035472 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29284
Aug 10 06:50:18 dist-test-slave-f7th krb5kdc[28644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344617, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:50:18.051311 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:18.036454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.22.11.131:39289)
0810 06:50:18.037117 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:18.037124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:18.037153 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:50:18.037216 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:18.037222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:18.037426 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:18.037586 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:18.038842 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:18.039941 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:18.041352 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:18.041898 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:18.042010 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:18.046145 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:18.046156 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:18.046169 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:18.046200 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:18.048730 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:18.049358 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:18.049365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:18.049367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:18.049496 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:18.049831 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:18.049834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:18.049836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:18.050164 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:18.050468 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:18.050744 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:50:18.050990 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":359,"threads_started":1}
I20260810 06:50:18.052524 29491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:18.133541 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29284
I20260810 06:50:18.158579 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28665
I20260810 06:50:18.159823 28763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:18.261397 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28665
2026-08-10T06:50:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3935 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:18 dist-test-slave-f7th krb5kdc[29524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:18 dist-test-slave-f7th krb5kdc[29524](info): set up 2 sockets
Aug 10 06:50:18 dist-test-slave-f7th krb5kdc[29524](info): commencing operation
krb5kdc: starting...
W20260810 06:50:20.229849 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.921s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:20 dist-test-slave-f7th krb5kdc[29524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:20.541587 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:42631
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:42631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:21.087360 29543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:21.087870 29543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:21.088169 29543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:21.099783 29543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:21.099936 29543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:21.100082 29543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:21.100217 29543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:21.100308 29543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:21.116636 29543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:42631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:42631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:21.118870 29543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:21.121052 29543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:21.139945 29551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:21.140022 29549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:21.140061 29553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:21.142386 29543 server_base.cc:1034] running on GCE node
I20260810 06:50:21.143589 29543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:21.145871 29543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:21.147156 29543 hybrid_clock.cc:648] HybridClock initialized: now 1786344621147098 us; error 53 us; skew 500 ppm
I20260810 06:50:21.147715 29543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:21 dist-test-slave-f7th krb5kdc[29524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:21.152926 29543 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:21.154708 29543 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:21.154892 29543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:21.155434 29543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:21.160606 29543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "eea4679002b7458f80b06812b123bfcb"
format_stamp: "Formatted at 2026-08-10 06:50:21 on dist-test-slave-f7th"
server_key: "ef12a8ac53f3bb0492f3ae0510e056c6"
server_key_iv: "406ed973d9ea92d20c9cc85b9ac1aebd"
server_key_version: "encryptionkey@0"
I20260810 06:50:21.161973 29543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "eea4679002b7458f80b06812b123bfcb"
format_stamp: "Formatted at 2026-08-10 06:50:21 on dist-test-slave-f7th"
server_key: "ef12a8ac53f3bb0492f3ae0510e056c6"
server_key_iv: "406ed973d9ea92d20c9cc85b9ac1aebd"
server_key_version: "encryptionkey@0"
I20260810 06:50:21.170799 29543 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260810 06:50:21.176561 29561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:21.178746 29543 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:50:21.178990 29543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "eea4679002b7458f80b06812b123bfcb"
format_stamp: "Formatted at 2026-08-10 06:50:21 on dist-test-slave-f7th"
server_key: "ef12a8ac53f3bb0492f3ae0510e056c6"
server_key_iv: "406ed973d9ea92d20c9cc85b9ac1aebd"
server_key_version: "encryptionkey@0"
I20260810 06:50:21.179325 29543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:21.200603 29543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:21.207345 29543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:21.207811 29543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:21.235755 29543 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:42631
I20260810 06:50:21.235778 29646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:42631 every 8 connection(s)
I20260810 06:50:21.237251 29543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:21.240311 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29543
I20260810 06:50:21.240695 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:21.241781 22574 external_mini_cluster.cc:1428] Setting key c538828679d9912eb8d9842f3aca7cec
I20260810 06:50:21.244360 29647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:50:21 dist-test-slave-f7th krb5kdc[29524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:21.268193 29647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb: Bootstrap starting.
I20260810 06:50:21.273823 29647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:21.275552 29647 log.cc:824] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:21.280102 29647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb: No bootstrap required, opened a new log
I20260810 06:50:21.281407 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:21.245219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44250 (local address 127.22.11.190:42631)
0810 06:50:21.246700 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:21.246733 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:21.246853 (+ 120us) server_negotiation.cc:407] Connection header received
0810 06:50:21.250237 (+ 3384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:21.250295 (+ 58us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:21.251380 (+ 1085us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:21.252185 (+ 805us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:21.255345 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:21.258561 (+ 3216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:21.260675 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:21.262558 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:21.262912 (+ 354us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:21.269446 (+ 6534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:21.269456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:21.269471 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:21.269542 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:21.274538 (+ 4996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:21.275613 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:21.275622 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:21.275637 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:21.275933 (+ 296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:21.276504 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:21.276510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:21.276512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:21.277449 (+ 937us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:21.277921 (+ 472us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:21.278728 (+ 807us) server_negotiation.cc:299] Negotiation successful
0810 06:50:21.279456 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1055,"thread_start_us":608,"threads_started":1}
I20260810 06:50:21.293125 29647 raft_consensus.cc:348] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4679002b7458f80b06812b123bfcb" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42631 } }
I20260810 06:50:21.293778 29647 raft_consensus.cc:374] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:21.293921 29647 raft_consensus.cc:729] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea4679002b7458f80b06812b123bfcb, State: Initialized, Role: FOLLOWER
I20260810 06:50:21.294785 29647 consensus_queue.cc:260] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4679002b7458f80b06812b123bfcb" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42631 } }
I20260810 06:50:21.295149 29647 raft_consensus.cc:388] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:21.295586 29647 raft_consensus.cc:482] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:21.295919 29647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:21.299765 29647 raft_consensus.cc:504] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4679002b7458f80b06812b123bfcb" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42631 } }
I20260810 06:50:21.300515 29647 leader_election.cc:302] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eea4679002b7458f80b06812b123bfcb; no voters:
I20260810 06:50:21.301414 29647 leader_election.cc:288] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:21.301592 29658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:21.303730 29658 raft_consensus.cc:686] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [term 1 LEADER]: Becoming Leader. State: Replica: eea4679002b7458f80b06812b123bfcb, State: Running, Role: LEADER
I20260810 06:50:21.304740 29658 consensus_queue.cc:237] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4679002b7458f80b06812b123bfcb" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42631 } }
I20260810 06:50:21.306123 29647 sys_catalog.cc:564] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:21.309787 29659 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eea4679002b7458f80b06812b123bfcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4679002b7458f80b06812b123bfcb" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42631 } } }
I20260810 06:50:21.310439 29659 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:21.311885 29661 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader eea4679002b7458f80b06812b123bfcb. Latest consensus state: current_term: 1 leader_uuid: "eea4679002b7458f80b06812b123bfcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4679002b7458f80b06812b123bfcb" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42631 } } }
I20260810 06:50:21.313397 29661 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:21.315028 29664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:21.323994 29664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:21.336891 29664 catalog_manager.cc:1347] Generated new cluster ID: 48dfbcac36b744b2946ecbcab62bac5b
I20260810 06:50:21.337078 29664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:21.384805 29664 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:21.388180 29664 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:21.403057 29664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb: Generated new TSK 0
I20260810 06:50:21.404438 29664 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:50:21.500947 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42631
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:21.891563 29694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:21.891830 29694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:21.891943 29694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:21.899019 29694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:21.899122 29694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:21.899242 29694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:21.909147 29694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:21.910346 29694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:21.911557 29694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:21.923651 29704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:21.923646 29702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:21.923676 29701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:21.924369 29694 server_base.cc:1034] running on GCE node
I20260810 06:50:21.925099 29694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:21.925984 29694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:21.927178 29694 hybrid_clock.cc:648] HybridClock initialized: now 1786344621927123 us; error 48 us; skew 500 ppm
I20260810 06:50:21.927466 29694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:21 dist-test-slave-f7th krb5kdc[29524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:21.930509 29694 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:50:21.931546 29694 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:21.931650 29694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:21.931946 29694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:21.934796 29694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "5dfa0a63d208498f85366a22ba929c52"
format_stamp: "Formatted at 2026-08-10 06:50:21 on dist-test-slave-f7th"
server_key: "71a17bb8ae81cce8b552df649560d91f"
server_key_iv: "016dba6b7cd6938cce60401c921e53ff"
server_key_version: "encryptionkey@0"
I20260810 06:50:21.935567 29694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "5dfa0a63d208498f85366a22ba929c52"
format_stamp: "Formatted at 2026-08-10 06:50:21 on dist-test-slave-f7th"
server_key: "71a17bb8ae81cce8b552df649560d91f"
server_key_iv: "016dba6b7cd6938cce60401c921e53ff"
server_key_version: "encryptionkey@0"
I20260810 06:50:21.940341 29694 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 06:50:21.943662 29714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:21.944990 29694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:21.945171 29694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "5dfa0a63d208498f85366a22ba929c52"
format_stamp: "Formatted at 2026-08-10 06:50:21 on dist-test-slave-f7th"
server_key: "71a17bb8ae81cce8b552df649560d91f"
server_key_iv: "016dba6b7cd6938cce60401c921e53ff"
server_key_version: "encryptionkey@0"
I20260810 06:50:21.945387 29694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:21.966392 29694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:21.970098 29694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:21.970319 29694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:21.971180 29694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:21.972455 29694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:21.972594 29694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:21.972707 29694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:21.972764 29694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:22.000816 29694 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:35377
I20260810 06:50:22.000883 29896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:35377 every 8 connection(s)
I20260810 06:50:22.001631 29694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:22.009222 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29694
I20260810 06:50:22.009426 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:22.010040 22574 external_mini_cluster.cc:1428] Setting key 5b8b519284abe6c29f78f54ebf4af335
Aug 10 06:50:22 dist-test-slave-f7th krb5kdc[29524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344621, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:22.021615 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:22.003958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:50059 (local address 127.22.11.190:42631)
0810 06:50:22.004396 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:22.004401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:22.005365 (+ 964us) server_negotiation.cc:407] Connection header received
0810 06:50:22.007277 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:22.007282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:22.007370 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:22.007464 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:22.009071 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.009856 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.011509 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.012027 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.012150 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:22.015704 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:22.015709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:22.015713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:22.015745 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:22.018379 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:22.019285 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:22.019289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:22.019291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:22.019414 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:22.019841 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:22.019844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:22.019846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:22.020180 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:22.020319 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:22.021015 (+ 696us) server_negotiation.cc:299] Negotiation successful
0810 06:50:22.021256 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":201,"threads_started":1}
I20260810 06:50:22.022487 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:22.004502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:42631 (local address 127.22.11.129:50059)
0810 06:50:22.005213 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:22.005251 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:22.007011 (+ 1760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:22.007716 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:22.007723 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:22.008270 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:22.008875 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:22.008894 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.009996 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.010000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:22.011363 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:22.011375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.012256 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.012264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:22.012467 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:22.013144 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:22.013174 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:22.015476 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:22.018513 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:22.018522 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:22.018538 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:22.019151 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:22.019513 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:22.019518 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:22.019521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:22.019748 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:22.020307 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:22.020313 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:22.020657 (+ 344us) client_negotiation.cc:769] Sending connection context
0810 06:50:22.020890 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 06:50:22.021280 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":276,"threads_started":1}
I20260810 06:50:22.023983 29899 heartbeater.cc:347] Connected to a master server at 127.22.11.190:42631
I20260810 06:50:22.024279 29899 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:22.024930 29899 heartbeater.cc:511] Master 127.22.11.190:42631 requested a full tablet report, sending...
I20260810 06:50:22.026954 29588 ts_manager.cc:194] Registered new tserver with Master: 5dfa0a63d208498f85366a22ba929c52 (127.22.11.129:35377)
I20260810 06:50:22.028373 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:50059
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:50:22.057250 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42631
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:22.378712 29909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:22.378968 29909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:22.379078 29909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:22.386026 29909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:22.386130 29909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:22.386277 29909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:22.395479 29909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:22.396636 29909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:22.397843 29909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:22.409924 29915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:22.409883 29916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:22.410121 29919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:22.410759 29909 server_base.cc:1034] running on GCE node
I20260810 06:50:22.411296 29909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:22.412143 29909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:22.413319 29909 hybrid_clock.cc:648] HybridClock initialized: now 1786344622413267 us; error 43 us; skew 500 ppm
I20260810 06:50:22.413609 29909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:22 dist-test-slave-f7th krb5kdc[29524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344622, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:22.416500 29909 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:50:22.417594 29909 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:22.417682 29909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:22.417989 29909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:22.420758 29909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "481c3fcf33714ebd9f208436d2ae2f66"
format_stamp: "Formatted at 2026-08-10 06:50:22 on dist-test-slave-f7th"
server_key: "b1a787433af94942e4fa704a40aa7122"
server_key_iv: "2e746221954381883e54ae255418e26d"
server_key_version: "encryptionkey@0"
I20260810 06:50:22.421489 29909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "481c3fcf33714ebd9f208436d2ae2f66"
format_stamp: "Formatted at 2026-08-10 06:50:22 on dist-test-slave-f7th"
server_key: "b1a787433af94942e4fa704a40aa7122"
server_key_iv: "2e746221954381883e54ae255418e26d"
server_key_version: "encryptionkey@0"
I20260810 06:50:22.426134 29909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:50:22.429339 29932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:22.430526 29909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:22.430667 29909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "481c3fcf33714ebd9f208436d2ae2f66"
format_stamp: "Formatted at 2026-08-10 06:50:22 on dist-test-slave-f7th"
server_key: "b1a787433af94942e4fa704a40aa7122"
server_key_iv: "2e746221954381883e54ae255418e26d"
server_key_version: "encryptionkey@0"
I20260810 06:50:22.430856 29909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:22.457562 29909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:22.461580 29909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:22.461829 29909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:22.462752 29909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:22.464049 29909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:22.464157 29909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:22.464270 29909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:22.464345 29909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:22.494125 29909 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:36485
I20260810 06:50:22.494172 30123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:36485 every 8 connection(s)
I20260810 06:50:22.494930 29909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:22.505215 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29909
I20260810 06:50:22.505399 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:22.505806 22574 external_mini_cluster.cc:1428] Setting key 9b8dad6910d36368ced05a606a805b08
Aug 10 06:50:22 dist-test-slave-f7th krb5kdc[29524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344622, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:22.513480 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:22.497303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:33811 (local address 127.22.11.190:42631)
0810 06:50:22.497486 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:22.497491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:22.498573 (+ 1082us) server_negotiation.cc:407] Connection header received
0810 06:50:22.500262 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:22.500266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:22.500334 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:22.500417 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:22.501803 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.502484 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.504246 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.504715 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.504827 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:22.508243 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:22.508248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:22.508252 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:22.508283 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:22.510664 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:22.511417 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:22.511420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:22.511423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:22.511554 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:22.511949 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:22.511952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:22.511954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:22.512185 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:22.512288 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:22.513065 (+ 777us) server_negotiation.cc:299] Negotiation successful
0810 06:50:22.513226 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:50:22.514688 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:22.497733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:42631 (local address 127.22.11.130:33811)
0810 06:50:22.498412 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:22.498458 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:22.500029 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:22.500592 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:22.500599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:22.501137 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:22.501672 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:22.501689 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.502614 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.502618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:22.504107 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:22.504120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:22.504899 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:22.504903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:22.505085 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:22.505683 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:22.505710 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:22.508050 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:22.510794 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:22.510802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:22.510814 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:22.511326 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:22.511646 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:22.511650 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:22.511653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:22.511864 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:22.512294 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:22.512300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:22.512713 (+ 413us) client_negotiation.cc:769] Sending connection context
0810 06:50:22.513007 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 06:50:22.513460 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":274,"threads_started":1}
I20260810 06:50:22.515980 30124 heartbeater.cc:347] Connected to a master server at 127.22.11.190:42631
I20260810 06:50:22.516276 30124 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:22.517040 30124 heartbeater.cc:511] Master 127.22.11.190:42631 requested a full tablet report, sending...
I20260810 06:50:22.518507 29588 ts_manager.cc:194] Registered new tserver with Master: 481c3fcf33714ebd9f208436d2ae2f66 (127.22.11.130:36485)
I20260810 06:50:22.519409 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:33811
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:50:22.551957 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42631
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:36699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:22.875978 30134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:22.876245 30134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:22.876370 30134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:22.883626 30134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:22.883751 30134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:22.883908 30134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:22.893430 30134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:36699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42631
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:22.894623 30134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:22.895905 30134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:22.908317 30151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:22.908190 30156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:22.908272 30152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:22.909281 30134 server_base.cc:1034] running on GCE node
I20260810 06:50:22.909809 30134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:22.910617 30134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:22.911804 30134 hybrid_clock.cc:648] HybridClock initialized: now 1786344622911762 us; error 35 us; skew 500 ppm
I20260810 06:50:22.912092 30134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:22 dist-test-slave-f7th krb5kdc[29524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344622, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:22.917826 30134 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:50:22.918941 30134 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:22.919045 30134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:22.919339 30134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:22.922325 30134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "d278b047d894413eb65eb8bd93710735"
format_stamp: "Formatted at 2026-08-10 06:50:22 on dist-test-slave-f7th"
server_key: "50cce9fd769bb842d44b08473be2d8ed"
server_key_iv: "c4b264aeadfa322bbd0a32e606759cd7"
server_key_version: "encryptionkey@0"
I20260810 06:50:22.923086 30134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "d278b047d894413eb65eb8bd93710735"
format_stamp: "Formatted at 2026-08-10 06:50:22 on dist-test-slave-f7th"
server_key: "50cce9fd769bb842d44b08473be2d8ed"
server_key_iv: "c4b264aeadfa322bbd0a32e606759cd7"
server_key_version: "encryptionkey@0"
I20260810 06:50:22.927929 30134 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 06:50:22.931205 30166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:22.932363 30134 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:50:22.932529 30134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "d278b047d894413eb65eb8bd93710735"
format_stamp: "Formatted at 2026-08-10 06:50:22 on dist-test-slave-f7th"
server_key: "50cce9fd769bb842d44b08473be2d8ed"
server_key_iv: "c4b264aeadfa322bbd0a32e606759cd7"
server_key_version: "encryptionkey@0"
I20260810 06:50:22.932767 30134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:22.987099 30134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:22.991289 30134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:22.991523 30134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:22.992420 30134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:22.993721 30134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:22.993813 30134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:22.993891 30134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:22.993947 30134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:23.024955 30134 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:43069
I20260810 06:50:23.025028 30371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:43069 every 8 connection(s)
I20260810 06:50:23.025753 30134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:23.031118 29899 heartbeater.cc:503] Master 127.22.11.190:42631 was elected leader, sending a full tablet report...
I20260810 06:50:23.031522 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 30134
I20260810 06:50:23.031720 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:23.032236 22574 external_mini_cluster.cc:1428] Setting key 7ae6c3d75cb19268fe61226d11c8f2c7
Aug 10 06:50:23 dist-test-slave-f7th krb5kdc[29524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344622, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:23.044695 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.028050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:39169 (local address 127.22.11.190:42631)
0810 06:50:23.028495 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.028499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.029432 (+ 933us) server_negotiation.cc:407] Connection header received
0810 06:50:23.031282 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.031288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.031377 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.031492 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:23.033284 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.034103 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.035736 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.036175 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.036286 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.039490 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:23.039494 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:23.039498 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:23.039528 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:23.041819 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.042489 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.042492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.042495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.042631 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.043040 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.043043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.043045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.043302 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:23.043438 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.044083 (+ 645us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.044308 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":214,"threads_started":1}
I20260810 06:50:23.045511 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.028574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:42631 (local address 127.22.11.131:39169)
0810 06:50:23.029278 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:23.029321 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:23.030955 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:23.031768 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:23.031779 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:23.032320 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:23.033111 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.033138 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.034252 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.034256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.035603 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.035615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.036407 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.036414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.036641 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.037242 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:23.037262 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:23.039284 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:23.041946 (+ 2662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:23.041954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:23.041968 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:23.042368 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:23.042734 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:23.042738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:23.042742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:23.042909 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:23.043427 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:23.043435 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:23.043731 (+ 296us) client_negotiation.cc:769] Sending connection context
0810 06:50:23.043943 (+ 212us) client_negotiation.cc:240] Negotiation successful
0810 06:50:23.044399 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":264,"threads_started":1}
I20260810 06:50:23.046759 30373 heartbeater.cc:347] Connected to a master server at 127.22.11.190:42631
I20260810 06:50:23.047037 30373 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:23.047618 30373 heartbeater.cc:511] Master 127.22.11.190:42631 requested a full tablet report, sending...
I20260810 06:50:23.048949 29588 ts_manager.cc:194] Registered new tserver with Master: d278b047d894413eb65eb8bd93710735 (127.22.11.131:43069)
I20260810 06:50:23.049710 29588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:39169
I20260810 06:50:23.059022 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:50:23.074539 30378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.063255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.22.11.190:42631)
0810 06:50:23.063413 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.063417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.063960 (+ 543us) server_negotiation.cc:407] Connection header received
0810 06:50:23.064089 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.064092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.064162 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.064237 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:23.065645 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.066332 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.067509 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.067892 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.067978 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.070051 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:23.070055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:23.070058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:23.070086 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:23.072460 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.073011 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.073014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.073016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.073136 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.073414 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.073416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.073418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.073700 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:23.073880 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.074104 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.074269 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:50:23.078088 29588 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:23.084738 29588 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:23.107425 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.096346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41718 (local address 127.22.11.129:35377)
0810 06:50:23.097850 (+ 1504us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.097858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.098555 (+ 697us) server_negotiation.cc:407] Connection header received
0810 06:50:23.098662 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.098670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.098889 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.099067 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:23.100022 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.101686 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.104850 (+ 3164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.106206 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.106328 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.106558 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.106889 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.107083 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332,"thread_start_us":1151,"threads_started":1}
I20260810 06:50:23.107573 30398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.095930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48938 (local address 127.22.11.131:43069)
0810 06:50:23.096667 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.096676 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.097361 (+ 685us) server_negotiation.cc:407] Connection header received
0810 06:50:23.097798 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.097806 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.098019 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.098139 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:23.099154 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.100844 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.104774 (+ 3930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.106206 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.106328 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.106548 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.107092 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.107245 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":1232,"threads_started":1}
I20260810 06:50:23.108357 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.095852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:43069 (local address 127.0.0.1:48938)
0810 06:50:23.097211 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:23.097263 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:23.097533 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:23.098463 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:23.098483 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:23.098516 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:23.098909 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.098925 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.101030 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.101036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.104551 (+ 3515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.104571 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.106690 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.106695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.106859 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.106896 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:50:23.107741 (+ 845us) client_negotiation.cc:240] Negotiation successful
0810 06:50:23.107902 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":1,"thread_start_us":1231,"threads_started":1}
I20260810 06:50:23.108484 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.095863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:35377 (local address 127.0.0.1:41718)
0810 06:50:23.098291 (+ 2428us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:23.098307 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:23.098414 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:23.099308 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:23.099314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:23.099340 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:23.099822 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.099835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.101866 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.101872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.104659 (+ 2787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.104671 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.106382 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.106389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.106611 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.106684 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 06:50:23.106815 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 06:50:23.107983 (+ 1168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2287,"mutex_wait_us":721,"thread_start_us":1298,"threads_started":1}
I20260810 06:50:23.111764 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.098354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:36485 (local address 127.0.0.1:34192)
0810 06:50:23.103255 (+ 4901us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:23.103271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:23.103946 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:23.104608 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:23.104646 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:23.104673 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:23.105002 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.105011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.106300 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.106306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.108547 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.108564 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.111021 (+ 2457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.111027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.111185 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.111216 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:50:23.111302 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:50:23.111416 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4780,"mutex_wait_us":25,"thread_start_us":4630,"threads_started":1}
I20260810 06:50:23.112100 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.098660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.22.11.130:36485)
0810 06:50:23.102390 (+ 3730us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.102396 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.103814 (+ 1418us) server_negotiation.cc:407] Connection header received
0810 06:50:23.104159 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.104167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.104390 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.104601 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:23.105153 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.106173 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.109437 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.110802 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.110943 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.111137 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.111433 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.111711 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3605,"thread_start_us":3491,"threads_started":1}
I20260810 06:50:23.111827 30244 tablet_service.cc:1467] Processing CreateTablet for tablet 2f589bdcd66c4261b47e08a5f4f252c3 (DEFAULT_TABLE table=test-table [id=0d35820485024ecfb52bf45ff88aecf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:23.113413 30244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f589bdcd66c4261b47e08a5f4f252c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:23.113059 29786 tablet_service.cc:1467] Processing CreateTablet for tablet 2f589bdcd66c4261b47e08a5f4f252c3 (DEFAULT_TABLE table=test-table [id=0d35820485024ecfb52bf45ff88aecf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:23.114444 29786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f589bdcd66c4261b47e08a5f4f252c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:23.115536 30014 tablet_service.cc:1467] Processing CreateTablet for tablet 2f589bdcd66c4261b47e08a5f4f252c3 (DEFAULT_TABLE table=test-table [id=0d35820485024ecfb52bf45ff88aecf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:23.116838 30014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f589bdcd66c4261b47e08a5f4f252c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:23.124496 30411 tablet_bootstrap.cc:492] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Bootstrap starting.
I20260810 06:50:23.128413 30411 tablet_bootstrap.cc:654] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:23.128448 30412 tablet_bootstrap.cc:492] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Bootstrap starting.
I20260810 06:50:23.129786 30411 log.cc:824] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:23.130240 30414 tablet_bootstrap.cc:492] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Bootstrap starting.
I20260810 06:50:23.131824 30412 tablet_bootstrap.cc:654] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:23.133060 30412 log.cc:824] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:23.133077 30414 tablet_bootstrap.cc:654] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:23.134222 30414 log.cc:824] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:23.135516 30411 tablet_bootstrap.cc:492] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: No bootstrap required, opened a new log
I20260810 06:50:23.135805 30411 ts_tablet_manager.cc:1397] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260810 06:50:23.138847 30412 tablet_bootstrap.cc:492] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: No bootstrap required, opened a new log
I20260810 06:50:23.139076 30412 ts_tablet_manager.cc:1397] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 06:50:23.139222 30414 tablet_bootstrap.cc:492] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: No bootstrap required, opened a new log
I20260810 06:50:23.139488 30414 ts_tablet_manager.cc:1397] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:50:23.145888 30412 raft_consensus.cc:348] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.146220 30412 raft_consensus.cc:374] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:23.146341 30412 raft_consensus.cc:729] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dfa0a63d208498f85366a22ba929c52, State: Initialized, Role: FOLLOWER
I20260810 06:50:23.146258 30411 raft_consensus.cc:348] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.146689 30411 raft_consensus.cc:374] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:23.146823 30411 raft_consensus.cc:729] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d278b047d894413eb65eb8bd93710735, State: Initialized, Role: FOLLOWER
I20260810 06:50:23.146958 30412 consensus_queue.cc:260] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.147555 30411 consensus_queue.cc:260] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.148783 30373 heartbeater.cc:503] Master 127.22.11.190:42631 was elected leader, sending a full tablet report...
I20260810 06:50:23.149583 30414 raft_consensus.cc:348] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.149962 30414 raft_consensus.cc:374] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:23.149973 30412 ts_tablet_manager.cc:1428] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Time spent starting tablet: real 0.011s user 0.009s sys 0.002s
I20260810 06:50:23.150090 30414 raft_consensus.cc:729] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481c3fcf33714ebd9f208436d2ae2f66, State: Initialized, Role: FOLLOWER
I20260810 06:50:23.151332 30411 ts_tablet_manager.cc:1428] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
I20260810 06:50:23.151686 30414 consensus_queue.cc:260] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.152861 30124 heartbeater.cc:503] Master 127.22.11.190:42631 was elected leader, sending a full tablet report...
I20260810 06:50:23.153371 30414 ts_tablet_manager.cc:1428] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260810 06:50:23.246812 30126 tablet.cc:2369] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:50:23.253597 29901 tablet.cc:2369] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:50:23.277135 30374 tablet.cc:2369] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:50:23.290061 30426 raft_consensus.cc:482] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:23.290333 30426 raft_consensus.cc:504] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.291951 30426 leader_election.cc:288] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d278b047d894413eb65eb8bd93710735 (127.22.11.131:43069), 5dfa0a63d208498f85366a22ba929c52 (127.22.11.129:35377)
I20260810 06:50:23.298960 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.292497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:53691 (local address 127.22.11.129:35377)
0810 06:50:23.292714 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.292720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.293273 (+ 553us) server_negotiation.cc:407] Connection header received
0810 06:50:23.293436 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.293440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.293508 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.293579 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:23.294195 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.294912 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.296723 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.297929 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.298070 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.298134 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.298594 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.298715 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:50:23.298955 30398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.292438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:46187 (local address 127.22.11.131:43069)
0810 06:50:23.292652 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.292659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.293122 (+ 463us) server_negotiation.cc:407] Connection header received
0810 06:50:23.293309 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.293313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.293373 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.293447 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:23.294073 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.294790 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.296647 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.297847 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.298012 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.298097 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.298587 (+ 490us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.298715 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260810 06:50:23.299206 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.292157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:43069 (local address 127.22.11.130:46187)
0810 06:50:23.293006 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:23.293021 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:23.293173 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:23.293476 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:23.293480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:23.293504 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:23.293920 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.293931 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.294920 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.294923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.296468 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.296480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.298115 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.298127 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.298337 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.298452 (+ 115us) client_negotiation.cc:769] Sending connection context
0810 06:50:23.298564 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:50:23.298714 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":152,"thread_start_us":517,"threads_started":1}
I20260810 06:50:23.299220 30428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.292364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:35377 (local address 127.22.11.130:53691)
0810 06:50:23.293175 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:23.293188 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:23.293311 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:23.293616 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:23.293620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:23.293658 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:23.294046 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.294056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.295056 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.295060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.296535 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:23.296548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.298138 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.298143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:23.298377 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.298452 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 06:50:23.298555 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:50:23.298714 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":113,"thread_start_us":528,"threads_started":1}
I20260810 06:50:23.300232 29822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f589bdcd66c4261b47e08a5f4f252c3" candidate_uuid: "481c3fcf33714ebd9f208436d2ae2f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dfa0a63d208498f85366a22ba929c52" is_pre_election: true
I20260810 06:50:23.300282 30288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f589bdcd66c4261b47e08a5f4f252c3" candidate_uuid: "481c3fcf33714ebd9f208436d2ae2f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d278b047d894413eb65eb8bd93710735" is_pre_election: true
I20260810 06:50:23.300753 29822 raft_consensus.cc:2454] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 481c3fcf33714ebd9f208436d2ae2f66 in term 0.
I20260810 06:50:23.300745 30288 raft_consensus.cc:2454] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 481c3fcf33714ebd9f208436d2ae2f66 in term 0.
I20260810 06:50:23.301355 29933 leader_election.cc:302] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 481c3fcf33714ebd9f208436d2ae2f66, 5dfa0a63d208498f85366a22ba929c52; no voters:
I20260810 06:50:23.301710 30426 raft_consensus.cc:2781] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:23.301834 30426 raft_consensus.cc:482] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:23.301927 30426 raft_consensus.cc:3037] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:23.304064 30426 raft_consensus.cc:504] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.304672 30426 leader_election.cc:288] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [CANDIDATE]: Term 1 election: Requested vote from peers d278b047d894413eb65eb8bd93710735 (127.22.11.131:43069), 5dfa0a63d208498f85366a22ba929c52 (127.22.11.129:35377)
I20260810 06:50:23.305066 30288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f589bdcd66c4261b47e08a5f4f252c3" candidate_uuid: "481c3fcf33714ebd9f208436d2ae2f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d278b047d894413eb65eb8bd93710735"
I20260810 06:50:23.305107 29822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f589bdcd66c4261b47e08a5f4f252c3" candidate_uuid: "481c3fcf33714ebd9f208436d2ae2f66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dfa0a63d208498f85366a22ba929c52"
I20260810 06:50:23.305279 30288 raft_consensus.cc:3037] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:23.305279 29822 raft_consensus.cc:3037] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:23.308408 30288 raft_consensus.cc:2454] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 481c3fcf33714ebd9f208436d2ae2f66 in term 1.
I20260810 06:50:23.308408 29822 raft_consensus.cc:2454] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 481c3fcf33714ebd9f208436d2ae2f66 in term 1.
I20260810 06:50:23.308868 29940 leader_election.cc:302] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 481c3fcf33714ebd9f208436d2ae2f66, 5dfa0a63d208498f85366a22ba929c52, d278b047d894413eb65eb8bd93710735; no voters:
I20260810 06:50:23.309186 30426 raft_consensus.cc:2781] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:23.309599 30426 raft_consensus.cc:686] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 1 LEADER]: Becoming Leader. State: Replica: 481c3fcf33714ebd9f208436d2ae2f66, State: Running, Role: LEADER
I20260810 06:50:23.310050 30426 consensus_queue.cc:237] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } }
I20260810 06:50:23.314138 29586 catalog_manager.cc:5495] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 reported cstate change: term changed from 0 to 1, leader changed from <none> to 481c3fcf33714ebd9f208436d2ae2f66 (127.22.11.130). New cstate: current_term: 1 leader_uuid: "481c3fcf33714ebd9f208436d2ae2f66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "481c3fcf33714ebd9f208436d2ae2f66" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36485 } health_report { overall_health: HEALTHY } } }
I20260810 06:50:23.342507 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.337890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34198 (local address 127.22.11.130:36485)
0810 06:50:23.338072 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.338078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.338203 (+ 125us) server_negotiation.cc:407] Connection header received
0810 06:50:23.338377 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.338383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.338478 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.338621 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:23.338901 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.339577 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.340813 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.341188 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.341270 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.341648 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:23.342113 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:23.342218 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.342251 (+ 33us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.342311 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:50:23.350252 29822 raft_consensus.cc:1261] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 1 FOLLOWER]: Refusing update from remote peer 481c3fcf33714ebd9f208436d2ae2f66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:23.350251 30288 raft_consensus.cc:1261] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 1 FOLLOWER]: Refusing update from remote peer 481c3fcf33714ebd9f208436d2ae2f66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:23.351271 30429 consensus_queue.cc:1035] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d278b047d894413eb65eb8bd93710735" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:23.351599 30426 consensus_queue.cc:1035] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dfa0a63d208498f85366a22ba929c52" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:23.366166 30437 mvcc.cc:205] Tried to move back new op lower bound from 7316867577232506880 to 7316867577081245696. Current Snapshot: MvccSnapshot[applied={T|T < 7316867577232506880}]
I20260810 06:50:23.375418 29586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:23.375644 29586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:23.379024 29586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eea4679002b7458f80b06812b123bfcb: Sending DeleteTablet for 3 replicas of tablet 2f589bdcd66c4261b47e08a5f4f252c3
I20260810 06:50:23.380040 30244 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f589bdcd66c4261b47e08a5f4f252c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:23 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:48938
I20260810 06:50:23.380512 29786 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f589bdcd66c4261b47e08a5f4f252c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:23 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:41718
I20260810 06:50:23.380735 30014 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f589bdcd66c4261b47e08a5f4f252c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:23 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:34192
I20260810 06:50:23.381278 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29694
I20260810 06:50:23.385181 30451 tablet_replica.cc:330] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: stopping tablet replica
I20260810 06:50:23.385711 30451 raft_consensus.cc:2229] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:23.386425 30451 raft_consensus.cc:2258] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:23.387271 30448 tablet_replica.cc:330] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: stopping tablet replica
I20260810 06:50:23.387648 30448 raft_consensus.cc:2229] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:23.388115 30448 raft_consensus.cc:2258] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:23.390084 30448 ts_tablet_manager.cc:1905] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:23.394281 30451 ts_tablet_manager.cc:1905] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:23.397087 30448 ts_tablet_manager.cc:1918] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:23.397321 30448 log.cc:1196] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/2f589bdcd66c4261b47e08a5f4f252c3
I20260810 06:50:23.397907 30448 ts_tablet_manager.cc:1939] T 2f589bdcd66c4261b47e08a5f4f252c3 P 5dfa0a63d208498f85366a22ba929c52: Deleting consensus metadata
I20260810 06:50:23.399816 29563 catalog_manager.cc:4841] TS 5dfa0a63d208498f85366a22ba929c52 (127.22.11.129:35377): tablet 2f589bdcd66c4261b47e08a5f4f252c3 (table test-table [id=0d35820485024ecfb52bf45ff88aecf6]) successfully deleted
I20260810 06:50:23.401161 30447 tablet_replica.cc:330] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: stopping tablet replica
I20260810 06:50:23.401592 30447 raft_consensus.cc:2229] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:23.402194 30447 raft_consensus.cc:2258] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:23.403031 30451 ts_tablet_manager.cc:1918] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:23.403268 30451 log.cc:1196] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/2f589bdcd66c4261b47e08a5f4f252c3
I20260810 06:50:23.403929 30451 ts_tablet_manager.cc:1939] T 2f589bdcd66c4261b47e08a5f4f252c3 P 481c3fcf33714ebd9f208436d2ae2f66: Deleting consensus metadata
I20260810 06:50:23.404060 30447 ts_tablet_manager.cc:1905] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:23.405567 29563 catalog_manager.cc:4841] TS 481c3fcf33714ebd9f208436d2ae2f66 (127.22.11.130:36485): tablet 2f589bdcd66c4261b47e08a5f4f252c3 (table test-table [id=0d35820485024ecfb52bf45ff88aecf6]) successfully deleted
Aug 10 06:50:23 dist-test-slave-f7th krb5kdc[29524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:23.411170 30447 ts_tablet_manager.cc:1918] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:23.411343 30447 log.cc:1196] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/2f589bdcd66c4261b47e08a5f4f252c3
I20260810 06:50:23.411859 30447 ts_tablet_manager.cc:1939] T 2f589bdcd66c4261b47e08a5f4f252c3 P d278b047d894413eb65eb8bd93710735: Deleting consensus metadata
I20260810 06:50:23.413642 29566 catalog_manager.cc:4841] TS d278b047d894413eb65eb8bd93710735 (127.22.11.131:43069): tablet 2f589bdcd66c4261b47e08a5f4f252c3 (table test-table [id=0d35820485024ecfb52bf45ff88aecf6]) successfully deleted
I20260810 06:50:23.417361 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.382390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.22.11.129:35377)
0810 06:50:23.382573 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.382580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.401126 (+ 18546us) server_negotiation.cc:407] Connection header received
0810 06:50:23.401305 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.401310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.401397 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.401512 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:23.403503 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.404588 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.406333 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.406706 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.406823 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.410882 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:23.410893 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:23.410906 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:23.410937 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:23.414717 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.415351 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.415358 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.415360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.415493 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.415846 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.415850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.415852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.416187 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:23.416488 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.416795 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.417009 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:50:23.418081 29887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:23.511281 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29694
I20260810 06:50:23.536455 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29909
Aug 10 06:50:23 dist-test-slave-f7th krb5kdc[29524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:50:23.555265 30408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.537296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.22.11.130:36485)
0810 06:50:23.537470 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.537479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.537534 (+ 55us) server_negotiation.cc:407] Connection header received
0810 06:50:23.537744 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.537748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.537830 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.537951 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:23.539532 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.540317 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.542155 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.542765 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.542889 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.548325 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:23.548339 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:23.548359 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:23.548406 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:23.552293 (+ 3887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.553133 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.553143 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.553146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.553316 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.553674 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.553678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.553681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.554137 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:23.554474 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.554642 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.554860 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:50:23.556010 30117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:23.669250 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29909
I20260810 06:50:23.694936 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 30134
Aug 10 06:50:23 dist-test-slave-f7th krb5kdc[29524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:50:23.709762 30398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:23.695709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.22.11.131:43069)
0810 06:50:23.695898 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:23.695903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:23.695924 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:50:23.696086 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:23.696091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:23.696157 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:23.696245 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:23.697565 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.698286 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.699534 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:23.699941 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:23.700068 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:23.704397 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:23.704412 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:23.704429 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:23.704470 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:23.707279 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.707889 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.707897 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.707899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.708051 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:23.708362 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:23.708365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:23.708367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:23.708852 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:23.709079 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:23.709324 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 06:50:23.709504 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:50:23.710376 30364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:23.797678 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 30134
I20260810 06:50:23.821488 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29543
I20260810 06:50:23.822570 29638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:23.930382 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29543
2026-08-10T06:50:23Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (5671 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T06:50:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:23Z Disabled control of system clock
I20260810 06:50:24.007403 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:46827
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:46827
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:50:24.319936 30467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:24.320202 30467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:24.320295 30467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:24.327329 30467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:24.327430 30467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:24.327473 30467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:24.327528 30467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:24.327570 30467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:24.336261 30467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:46827
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:46827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:24.337405 30467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:24.339231 30467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:24.350168 30477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:24.350294 30474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:24.350215 30482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:24.350858 30467 server_base.cc:1034] running on GCE node
I20260810 06:50:24.351541 30467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:24.352782 30467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:24.353996 30467 hybrid_clock.cc:648] HybridClock initialized: now 1786344624353950 us; error 41 us; skew 500 ppm
I20260810 06:50:24.354318 30467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:24.356357 30467 webserver.cc:460] Webserver started at http://127.22.11.190:43923/ using document root <none> and password file <none>
I20260810 06:50:24.357023 30467 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:24.357123 30467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:24.357421 30467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:24.360121 30467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6"
format_stamp: "Formatted at 2026-08-10 06:50:24 on dist-test-slave-f7th"
server_key: "387632f0008682d52e1a50d68a64b90f"
server_key_iv: "1cb3f1279b3f5d13111b9a732a057731"
server_key_version: "encryptionkey@0"
I20260810 06:50:24.360908 30467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6"
format_stamp: "Formatted at 2026-08-10 06:50:24 on dist-test-slave-f7th"
server_key: "387632f0008682d52e1a50d68a64b90f"
server_key_iv: "1cb3f1279b3f5d13111b9a732a057731"
server_key_version: "encryptionkey@0"
I20260810 06:50:24.365721 30467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:50:24.368860 30496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:24.370029 30467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:50:24.370165 30467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6"
format_stamp: "Formatted at 2026-08-10 06:50:24 on dist-test-slave-f7th"
server_key: "387632f0008682d52e1a50d68a64b90f"
server_key_iv: "1cb3f1279b3f5d13111b9a732a057731"
server_key_version: "encryptionkey@0"
I20260810 06:50:24.370348 30467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:24.375772 30467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:24.376451 30467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:24.376752 30467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:24.393217 30467 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:46827
I20260810 06:50:24.393249 30585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:46827 every 8 connection(s)
I20260810 06:50:24.394423 30467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:24.397915 30588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:24.403882 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 30467
I20260810 06:50:24.404060 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:24.404732 22574 external_mini_cluster.cc:1428] Setting key 125c18da2aaca8ff04307afca04e9325
I20260810 06:50:24.408179 30588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6: Bootstrap starting.
I20260810 06:50:24.411199 30588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:24.412276 30588 log.cc:824] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:24.412464 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:24.406596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44474 (local address 127.22.11.190:46827)
0810 06:50:24.407504 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:24.407514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:24.407553 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:50:24.409195 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:24.409217 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:24.409626 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:24.409968 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:24.410993 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:24.411001 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:24.411013 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:24.411028 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:24.411086 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:24.411101 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:24.411271 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:24.411403 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:50:24.411659 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":325,"threads_started":1}
I20260810 06:50:24.415036 30588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6: No bootstrap required, opened a new log
I20260810 06:50:24.421895 30588 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46827 } }
I20260810 06:50:24.422199 30588 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:24.422289 30588 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd4e8b7b426f41cb9b82b5c0016eb4c6, State: Initialized, Role: FOLLOWER
I20260810 06:50:24.422803 30588 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46827 } }
I20260810 06:50:24.423005 30588 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:24.423130 30588 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:24.423266 30588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:24.425534 30588 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46827 } }
I20260810 06:50:24.425992 30588 leader_election.cc:302] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd4e8b7b426f41cb9b82b5c0016eb4c6; no voters:
I20260810 06:50:24.426533 30588 leader_election.cc:288] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:24.426636 30595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:24.427270 30595 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [term 1 LEADER]: Becoming Leader. State: Replica: fd4e8b7b426f41cb9b82b5c0016eb4c6, State: Running, Role: LEADER
I20260810 06:50:24.427772 30595 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46827 } }
I20260810 06:50:24.428416 30588 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:24.430200 30596 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46827 } } }
I20260810 06:50:24.430384 30599 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd4e8b7b426f41cb9b82b5c0016eb4c6. Latest consensus state: current_term: 1 leader_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd4e8b7b426f41cb9b82b5c0016eb4c6" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 46827 } } }
I20260810 06:50:24.430603 30596 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:24.430673 30599 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:24.432384 30607 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:24.436364 30607 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:24.444341 30607 catalog_manager.cc:1347] Generated new cluster ID: edc76b96cc9c4897a2dc09a8ee08bcbc
I20260810 06:50:24.444446 30607 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:24.456362 30607 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:24.456861 30607 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:24.463948 30607 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6: Generated new TSK 0
I20260810 06:50:24.464886 30607 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:50:24.473671 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:46827
--builtin_ntp_servers=127.22.11.148:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 06:50:24.802557 30625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:24.802848 30625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:24.802987 30625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:24.810272 30625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:24.810385 30625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:24.810555 30625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:24.819587 30625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:46827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:24.820914 30625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:24.822698 30625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:24.835561 30634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:24.835688 30637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:24.835597 30632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:24.836583 30625 server_base.cc:1034] running on GCE node
I20260810 06:50:24.837239 30625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:24.838171 30625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:24.839385 30625 hybrid_clock.cc:648] HybridClock initialized: now 1786344624839322 us; error 50 us; skew 500 ppm
I20260810 06:50:24.839704 30625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:24.841969 30625 webserver.cc:460] Webserver started at http://127.22.11.129:45467/ using document root <none> and password file <none>
I20260810 06:50:24.842654 30625 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:24.842754 30625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:24.843083 30625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:24.845978 30625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "3e47702bc75349808961ff2807baa860"
format_stamp: "Formatted at 2026-08-10 06:50:24 on dist-test-slave-f7th"
server_key: "0e8c2f48a51eb515f95f55baf07bcdb6"
server_key_iv: "18e34ac01f0efbeed51a6362cd627874"
server_key_version: "encryptionkey@0"
I20260810 06:50:24.846741 30625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "3e47702bc75349808961ff2807baa860"
format_stamp: "Formatted at 2026-08-10 06:50:24 on dist-test-slave-f7th"
server_key: "0e8c2f48a51eb515f95f55baf07bcdb6"
server_key_iv: "18e34ac01f0efbeed51a6362cd627874"
server_key_version: "encryptionkey@0"
I20260810 06:50:24.852013 30625 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:50:24.855477 30652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:24.856876 30625 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:50:24.857043 30625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "3e47702bc75349808961ff2807baa860"
format_stamp: "Formatted at 2026-08-10 06:50:24 on dist-test-slave-f7th"
server_key: "0e8c2f48a51eb515f95f55baf07bcdb6"
server_key_iv: "18e34ac01f0efbeed51a6362cd627874"
server_key_version: "encryptionkey@0"
I20260810 06:50:24.857252 30625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:24.862833 30625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:24.863590 30625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:24.863838 30625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:24.864924 30625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:24.866346 30625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:24.866451 30625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:24.866593 30625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:24.866691 30625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:24.896729 30625 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:41531
I20260810 06:50:24.896773 30850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:41531 every 8 connection(s)
I20260810 06:50:24.898042 30625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:24.906075 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:24.900780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:38515 (local address 127.22.11.190:46827)
0810 06:50:24.900985 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:24.900991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:24.902400 (+ 1409us) server_negotiation.cc:407] Connection header received
0810 06:50:24.904067 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:24.904071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:24.904128 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:24.904219 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:24.905474 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:24.905478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:24.905482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:24.905498 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:24.905528 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:24.905540 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:24.905643 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:24.905826 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 06:50:24.905875 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:50:24.906102 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 30625
I20260810 06:50:24.906286 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:24.907163 22574 external_mini_cluster.cc:1428] Setting key 24a605628f349f3fd3757f90da51e79c
I20260810 06:50:24.907428 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:24.901341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:46827 (local address 127.22.11.129:38515)
0810 06:50:24.902248 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:24.902284 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:24.903819 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:24.904415 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:24.904423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:24.904508 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:24.905186 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:24.905188 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:24.905226 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:24.905315 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:24.905627 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:24.905636 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:24.905644 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:50:24.905927 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 06:50:24.906226 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":257,"threads_started":1}
I20260810 06:50:24.908931 30851 heartbeater.cc:347] Connected to a master server at 127.22.11.190:46827
I20260810 06:50:24.909251 30851 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:24.909864 30851 heartbeater.cc:511] Master 127.22.11.190:46827 requested a full tablet report, sending...
I20260810 06:50:24.910272 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:46827
--builtin_ntp_servers=127.22.11.148:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:50:24.911808 30527 ts_manager.cc:194] Registered new tserver with Master: 3e47702bc75349808961ff2807baa860 (127.22.11.129:41531)
W20260810 06:50:25.240758 30856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:25.241047 30856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:25.241179 30856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:25.249763 30856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:25.249945 30856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:25.250139 30856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:25.262780 30856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:46827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:25.264086 30856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:25.265919 30856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:25.278367 30872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:25.278481 30863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:25.278434 30865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:25.279464 30856 server_base.cc:1034] running on GCE node
I20260810 06:50:25.280099 30856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:25.281035 30856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:25.282235 30856 hybrid_clock.cc:648] HybridClock initialized: now 1786344625282186 us; error 41 us; skew 500 ppm
I20260810 06:50:25.282537 30856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:25.284716 30856 webserver.cc:460] Webserver started at http://127.22.11.130:43483/ using document root <none> and password file <none>
I20260810 06:50:25.285382 30856 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:25.285487 30856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:25.285804 30856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:25.288590 30856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "144142cea6344fdca0cdcd2b87211c2e"
format_stamp: "Formatted at 2026-08-10 06:50:25 on dist-test-slave-f7th"
server_key: "4f7914558af9e6b9d3ee84d1fa5f760e"
server_key_iv: "556b911a239e76f6f6d9634a33b691f9"
server_key_version: "encryptionkey@0"
I20260810 06:50:25.289415 30856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "144142cea6344fdca0cdcd2b87211c2e"
format_stamp: "Formatted at 2026-08-10 06:50:25 on dist-test-slave-f7th"
server_key: "4f7914558af9e6b9d3ee84d1fa5f760e"
server_key_iv: "556b911a239e76f6f6d9634a33b691f9"
server_key_version: "encryptionkey@0"
I20260810 06:50:25.294262 30856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:50:25.297634 30882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:25.298857 30856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:50:25.299007 30856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "144142cea6344fdca0cdcd2b87211c2e"
format_stamp: "Formatted at 2026-08-10 06:50:25 on dist-test-slave-f7th"
server_key: "4f7914558af9e6b9d3ee84d1fa5f760e"
server_key_iv: "556b911a239e76f6f6d9634a33b691f9"
server_key_version: "encryptionkey@0"
I20260810 06:50:25.299199 30856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:25.304596 30856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:25.305414 30856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:25.305658 30856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:25.306524 30856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:25.307890 30856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:25.307986 30856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:25.308094 30856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:25.308164 30856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:25.337917 30856 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:46519
I20260810 06:50:25.337977 31086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:46519 every 8 connection(s)
I20260810 06:50:25.339145 30856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:25.344892 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 30856
I20260810 06:50:25.345044 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:25.345479 22574 external_mini_cluster.cc:1428] Setting key 65533e7fa0d3cc93f9c4aefbd0755c24
I20260810 06:50:25.346923 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.341585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:43023 (local address 127.22.11.190:46827)
0810 06:50:25.341759 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.341764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.343063 (+ 1299us) server_negotiation.cc:407] Connection header received
0810 06:50:25.344684 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.344690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.344754 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.344846 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.346123 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.346127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.346130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.346146 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.346179 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.346191 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.346313 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.346527 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.346587 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:50:25.347755 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.342097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:46827 (local address 127.22.11.130:43023)
0810 06:50:25.342940 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:25.342979 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:25.344390 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:25.345089 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:25.345099 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:25.345197 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:25.345816 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:25.345818 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:25.345853 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:25.345946 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:25.346295 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:25.346306 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:25.346314 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:50:25.346554 (+ 240us) client_negotiation.cc:240] Negotiation successful
0810 06:50:25.346731 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":249,"threads_started":1}
I20260810 06:50:25.348380 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:46827
--builtin_ntp_servers=127.22.11.148:43509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 06:50:25.349063 31087 heartbeater.cc:347] Connected to a master server at 127.22.11.190:46827
I20260810 06:50:25.349329 31087 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:25.349851 31087 heartbeater.cc:511] Master 127.22.11.190:46827 requested a full tablet report, sending...
I20260810 06:50:25.351167 30527 ts_manager.cc:194] Registered new tserver with Master: 144142cea6344fdca0cdcd2b87211c2e (127.22.11.130:46519)
W20260810 06:50:25.671221 31091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:25.671492 31091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:25.671590 31091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:25.678678 31091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:25.678769 31091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:25.678884 31091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:25.687685 31091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:46827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:25.688937 31091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:25.690630 31091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:25.703639 31097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:25.703553 31098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:25.703594 31100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:25.704270 31091 server_base.cc:1034] running on GCE node
I20260810 06:50:25.704881 31091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:25.705708 31091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:25.706913 31091 hybrid_clock.cc:648] HybridClock initialized: now 1786344625706856 us; error 49 us; skew 500 ppm
I20260810 06:50:25.707207 31091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:25.709378 31091 webserver.cc:460] Webserver started at http://127.22.11.131:33203/ using document root <none> and password file <none>
I20260810 06:50:25.710062 31091 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:25.710209 31091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:25.710526 31091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:25.713249 31091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc"
format_stamp: "Formatted at 2026-08-10 06:50:25 on dist-test-slave-f7th"
server_key: "db98fa0575e57c84d1a8977e0520ebfa"
server_key_iv: "016dc1fa14d92c06511d4ee6844474ad"
server_key_version: "encryptionkey@0"
I20260810 06:50:25.713990 31091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc"
format_stamp: "Formatted at 2026-08-10 06:50:25 on dist-test-slave-f7th"
server_key: "db98fa0575e57c84d1a8977e0520ebfa"
server_key_iv: "016dc1fa14d92c06511d4ee6844474ad"
server_key_version: "encryptionkey@0"
I20260810 06:50:25.718726 31091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:50:25.721877 31107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:25.723062 31091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260810 06:50:25.723208 31091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc"
format_stamp: "Formatted at 2026-08-10 06:50:25 on dist-test-slave-f7th"
server_key: "db98fa0575e57c84d1a8977e0520ebfa"
server_key_iv: "016dc1fa14d92c06511d4ee6844474ad"
server_key_version: "encryptionkey@0"
I20260810 06:50:25.723392 31091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:25.728597 31091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:25.729413 31091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:25.729655 31091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:25.730501 31091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:25.731791 31091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:25.731884 31091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:25.731999 31091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:25.732069 31091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:25.759982 31091 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:36759
I20260810 06:50:25.760044 31314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:36759 every 8 connection(s)
I20260810 06:50:25.761291 31091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:25.768656 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.763727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:39015 (local address 127.22.11.190:46827)
0810 06:50:25.763956 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.763962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.765277 (+ 1315us) server_negotiation.cc:407] Connection header received
0810 06:50:25.766870 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.766874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.766927 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.767019 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.768113 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.768117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.768120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.768135 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.768164 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.768176 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.768264 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.768393 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.768441 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:50:25.769198 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 31091
I20260810 06:50:25.769367 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:25.769783 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.764239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:46827 (local address 127.22.11.131:39015)
0810 06:50:25.765129 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:25.765171 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:25.766643 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:25.767202 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:25.767210 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:25.767292 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:25.767858 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:25.767860 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:25.767894 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:25.767972 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:25.768246 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:25.768251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:25.768256 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:50:25.768450 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 06:50:25.768654 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":250,"threads_started":1}
I20260810 06:50:25.769919 22574 external_mini_cluster.cc:1428] Setting key f1b2d02f5fcf56aefb82bd542f0ac1d0
I20260810 06:50:25.771180 31315 heartbeater.cc:347] Connected to a master server at 127.22.11.190:46827
I20260810 06:50:25.771443 31315 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:25.772055 31315 heartbeater.cc:511] Master 127.22.11.190:46827 requested a full tablet report, sending...
I20260810 06:50:25.773437 30527 ts_manager.cc:194] Registered new tserver with Master: f5457f8dcf4c427c9a0dace6ec7d87cc (127.22.11.131:36759)
I20260810 06:50:25.784760 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:50:25.793032 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.788930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.22.11.190:46827)
0810 06:50:25.789105 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.789110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.791490 (+ 2380us) server_negotiation.cc:407] Connection header received
0810 06:50:25.791631 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.791635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.791683 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.791749 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.792551 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.792555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.792559 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.792573 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.792602 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.792636 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.792731 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.792782 (+ 51us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.792825 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:50:25.796495 30527 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:25.802958 30527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:25.820670 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.814999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.22.11.130:46519)
0810 06:50:25.816654 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.816663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.817127 (+ 464us) server_negotiation.cc:407] Connection header received
0810 06:50:25.817381 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.817390 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.817616 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.817748 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.819584 (+ 1836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.819593 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.819607 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.819633 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.819695 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.819718 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.819881 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.820168 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.820267 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1490,"thread_start_us":1377,"threads_started":1}
I20260810 06:50:25.821564 31338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.814384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:46519 (local address 127.0.0.1:39970)
0810 06:50:25.817002 (+ 2618us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:25.817034 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:25.817218 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:25.818643 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:25.818648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:25.818709 (+ 61us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:25.819339 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:25.819341 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:25.819362 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:25.819419 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:25.819938 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:25.819950 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:25.819958 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:50:25.821053 (+ 1095us) client_negotiation.cc:240] Negotiation successful
0810 06:50:25.821150 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2521,"thread_start_us":1227,"threads_started":1}
I20260810 06:50:25.823412 31346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.817903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:36759 (local address 127.0.0.1:40906)
0810 06:50:25.819662 (+ 1759us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:25.819679 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:25.819760 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:25.821585 (+ 1825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:25.821590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:25.821617 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:25.822276 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:25.822279 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:25.822295 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:25.822336 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:25.822837 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:25.822841 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:25.822846 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:50:25.822975 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 06:50:25.823014 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1655,"thread_start_us":1207,"threads_started":1}
I20260810 06:50:25.823963 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.817468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.22.11.131:36759)
0810 06:50:25.820291 (+ 2823us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.820298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.820328 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:50:25.820388 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.820396 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.820585 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.820756 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.822505 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.822513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.822526 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.822545 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.822601 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.822622 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.822781 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.823613 (+ 832us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.823700 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2661,"thread_start_us":2481,"threads_started":1}
I20260810 06:50:25.825697 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.814249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60084)
0810 06:50:25.815988 (+ 1739us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:25.816041 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:25.816160 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:25.822567 (+ 6407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:25.822571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:25.822593 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:25.823201 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:25.823204 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:25.823220 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:25.823264 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:25.825135 (+ 1871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:25.825140 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:25.825145 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:50:25.825245 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:50:25.825292 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1494,"thread_start_us":2531,"threads_started":1}
I20260810 06:50:25.825491 30962 tablet_service.cc:1467] Processing CreateTablet for tablet 2c14272f7d324628a405c5ebd73eca49 (DEFAULT_TABLE table=test-table [id=2b730ececbcb4bbea02da4db6fa0d47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:25.826437 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.814481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.22.11.129:41531)
0810 06:50:25.822061 (+ 7580us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.822078 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.822104 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:50:25.822168 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.822176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.822363 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.822554 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.824855 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.824863 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.824875 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.824901 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.824958 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.824979 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.825947 (+ 968us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.826023 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.826131 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7411,"thread_start_us":7284,"threads_started":1}
I20260810 06:50:25.826962 30962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c14272f7d324628a405c5ebd73eca49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:25.827630 31198 tablet_service.cc:1467] Processing CreateTablet for tablet 2c14272f7d324628a405c5ebd73eca49 (DEFAULT_TABLE table=test-table [id=2b730ececbcb4bbea02da4db6fa0d47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:25.828814 31198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c14272f7d324628a405c5ebd73eca49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:25.830160 30732 tablet_service.cc:1467] Processing CreateTablet for tablet 2c14272f7d324628a405c5ebd73eca49 (DEFAULT_TABLE table=test-table [id=2b730ececbcb4bbea02da4db6fa0d47f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:25.831449 30732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c14272f7d324628a405c5ebd73eca49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:25.839425 31347 tablet_bootstrap.cc:492] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Bootstrap starting.
I20260810 06:50:25.842569 31347 tablet_bootstrap.cc:654] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:25.843850 31347 log.cc:824] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:25.846052 31356 tablet_bootstrap.cc:492] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: Bootstrap starting.
I20260810 06:50:25.847071 31347 tablet_bootstrap.cc:492] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: No bootstrap required, opened a new log
I20260810 06:50:25.847304 31347 ts_tablet_manager.cc:1397] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:50:25.848716 31350 tablet_bootstrap.cc:492] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Bootstrap starting.
I20260810 06:50:25.848834 31356 tablet_bootstrap.cc:654] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:25.849922 31356 log.cc:824] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:25.852073 31350 tablet_bootstrap.cc:654] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:25.853302 31350 log.cc:824] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:25.854590 31356 tablet_bootstrap.cc:492] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: No bootstrap required, opened a new log
I20260810 06:50:25.854825 31356 ts_tablet_manager.cc:1397] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260810 06:50:25.856122 31350 tablet_bootstrap.cc:492] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: No bootstrap required, opened a new log
I20260810 06:50:25.856377 31350 ts_tablet_manager.cc:1397] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260810 06:50:25.856463 31347 raft_consensus.cc:348] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.856927 31347 raft_consensus.cc:374] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:25.857048 31347 raft_consensus.cc:729] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5457f8dcf4c427c9a0dace6ec7d87cc, State: Initialized, Role: FOLLOWER
I20260810 06:50:25.857650 31347 consensus_queue.cc:260] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.859862 31315 heartbeater.cc:503] Master 127.22.11.190:46827 was elected leader, sending a full tablet report...
I20260810 06:50:25.860533 31347 ts_tablet_manager.cc:1428] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 06:50:25.864756 31350 raft_consensus.cc:348] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.865075 31350 raft_consensus.cc:374] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:25.865170 31350 raft_consensus.cc:729] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 144142cea6344fdca0cdcd2b87211c2e, State: Initialized, Role: FOLLOWER
I20260810 06:50:25.865427 31356 raft_consensus.cc:348] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.865823 31356 raft_consensus.cc:374] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:25.865747 31350 consensus_queue.cc:260] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.865918 31356 raft_consensus.cc:729] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e47702bc75349808961ff2807baa860, State: Initialized, Role: FOLLOWER
I20260810 06:50:25.866482 31356 consensus_queue.cc:260] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.866783 31087 heartbeater.cc:503] Master 127.22.11.190:46827 was elected leader, sending a full tablet report...
I20260810 06:50:25.867322 31350 ts_tablet_manager.cc:1428] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260810 06:50:25.867513 30851 heartbeater.cc:503] Master 127.22.11.190:46827 was elected leader, sending a full tablet report...
I20260810 06:50:25.868093 31356 ts_tablet_manager.cc:1428] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: Time spent starting tablet: real 0.013s user 0.006s sys 0.006s
W20260810 06:50:25.900205 30852 tablet.cc:2369] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:50:25.915127 31364 raft_consensus.cc:482] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:25.915393 31364 raft_consensus.cc:504] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.917079 31364 leader_election.cc:288] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e47702bc75349808961ff2807baa860 (127.22.11.129:41531), f5457f8dcf4c427c9a0dace6ec7d87cc (127.22.11.131:36759)
I20260810 06:50:25.920231 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.917423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:52033 (local address 127.22.11.129:41531)
0810 06:50:25.917590 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.917596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.918186 (+ 590us) server_negotiation.cc:407] Connection header received
0810 06:50:25.918333 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.918338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.918400 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.918503 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.919597 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.919603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.919608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.919630 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.919671 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.919687 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.919775 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.919901 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.919961 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:50:25.920382 31368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.917432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.22.11.130:52033)
0810 06:50:25.918085 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:25.918100 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:25.918217 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:25.918654 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:25.918658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:25.918679 (+ 21us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:25.919420 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:25.919422 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:25.919436 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:25.919474 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:25.919776 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:25.919781 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:25.919786 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:50:25.919935 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 06:50:25.919976 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":363,"threads_started":1}
I20260810 06:50:25.920531 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.918384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:34473 (local address 127.22.11.131:36759)
0810 06:50:25.918555 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.918561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.918838 (+ 277us) server_negotiation.cc:407] Connection header received
0810 06:50:25.918945 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.918951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.919010 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.919098 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.919942 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.919948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.919952 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.919969 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.920006 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.920020 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.920140 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.920239 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.920301 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:50:25.920910 31369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.918156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:36759 (local address 127.22.11.130:34473)
0810 06:50:25.918733 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:25.918749 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:25.918843 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:25.919154 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:25.919158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:25.919180 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:25.919750 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:50:25.919753 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:25.919768 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:50:25.919805 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:25.920124 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:25.920129 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:25.920134 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:50:25.920282 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 06:50:25.920337 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":269,"threads_started":1}
I20260810 06:50:25.921433 30763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c14272f7d324628a405c5ebd73eca49" candidate_uuid: "144142cea6344fdca0cdcd2b87211c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e47702bc75349808961ff2807baa860" is_pre_election: true
I20260810 06:50:25.921881 30763 raft_consensus.cc:2454] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144142cea6344fdca0cdcd2b87211c2e in term 0.
I20260810 06:50:25.921761 31234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c14272f7d324628a405c5ebd73eca49" candidate_uuid: "144142cea6344fdca0cdcd2b87211c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" is_pre_election: true
I20260810 06:50:25.922199 31234 raft_consensus.cc:2454] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 144142cea6344fdca0cdcd2b87211c2e in term 0.
I20260810 06:50:25.922490 30890 leader_election.cc:302] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144142cea6344fdca0cdcd2b87211c2e, 3e47702bc75349808961ff2807baa860; no voters:
I20260810 06:50:25.922854 31364 raft_consensus.cc:2781] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:25.922984 31364 raft_consensus.cc:482] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:25.923080 31364 raft_consensus.cc:3037] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:25.925398 31364 raft_consensus.cc:504] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.925899 31364 leader_election.cc:288] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [CANDIDATE]: Term 1 election: Requested vote from peers 3e47702bc75349808961ff2807baa860 (127.22.11.129:41531), f5457f8dcf4c427c9a0dace6ec7d87cc (127.22.11.131:36759)
I20260810 06:50:25.926324 30763 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c14272f7d324628a405c5ebd73eca49" candidate_uuid: "144142cea6344fdca0cdcd2b87211c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e47702bc75349808961ff2807baa860"
I20260810 06:50:25.926368 31234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c14272f7d324628a405c5ebd73eca49" candidate_uuid: "144142cea6344fdca0cdcd2b87211c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc"
I20260810 06:50:25.926527 30763 raft_consensus.cc:3037] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:25.926533 31234 raft_consensus.cc:3037] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:25.929729 30763 raft_consensus.cc:2454] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144142cea6344fdca0cdcd2b87211c2e in term 1.
I20260810 06:50:25.929726 31234 raft_consensus.cc:2454] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 144142cea6344fdca0cdcd2b87211c2e in term 1.
I20260810 06:50:25.930069 30890 leader_election.cc:302] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 144142cea6344fdca0cdcd2b87211c2e, f5457f8dcf4c427c9a0dace6ec7d87cc; no voters:
I20260810 06:50:25.930313 31364 raft_consensus.cc:2781] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:25.930717 31364 raft_consensus.cc:686] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 1 LEADER]: Becoming Leader. State: Replica: 144142cea6344fdca0cdcd2b87211c2e, State: Running, Role: LEADER
I20260810 06:50:25.931141 31364 consensus_queue.cc:237] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } }
I20260810 06:50:25.935338 30527 catalog_manager.cc:5495] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 144142cea6344fdca0cdcd2b87211c2e (127.22.11.130). New cstate: current_term: 1 leader_uuid: "144142cea6344fdca0cdcd2b87211c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "144142cea6344fdca0cdcd2b87211c2e" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 46519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 } health_report { overall_health: UNKNOWN } } }
I20260810 06:50:25.948923 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.947146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.22.11.130:46519)
0810 06:50:25.947310 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.947314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.947330 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:50:25.947506 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.947510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.947581 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.947660 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:25.948434 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:25.948439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:25.948442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:25.948457 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:25.948487 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:25.948498 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:25.948603 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:25.948701 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 06:50:25.948749 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:50:25.956773 31234 raft_consensus.cc:1261] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 1 FOLLOWER]: Refusing update from remote peer 144142cea6344fdca0cdcd2b87211c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:25.956768 30763 raft_consensus.cc:1261] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 1 FOLLOWER]: Refusing update from remote peer 144142cea6344fdca0cdcd2b87211c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:25.957687 31370 consensus_queue.cc:1035] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e47702bc75349808961ff2807baa860" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:25.958060 31364 consensus_queue.cc:1035] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5457f8dcf4c427c9a0dace6ec7d87cc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:25.967832 31373 mvcc.cc:205] Tried to move back new op lower bound from 7316867587908878336 to 7316867587817504768. Current Snapshot: MvccSnapshot[applied={T|T < 7316867587908878336}]
I20260810 06:50:25.970988 31377 mvcc.cc:205] Tried to move back new op lower bound from 7316867587908878336 to 7316867587817504768. Current Snapshot: MvccSnapshot[applied={T|T < 7316867587908878336}]
I20260810 06:50:25.973554 31375 mvcc.cc:205] Tried to move back new op lower bound from 7316867587908878336 to 7316867587817504768. Current Snapshot: MvccSnapshot[applied={T|T < 7316867587908878336}]
I20260810 06:50:25.984768 30528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:25.985020 30528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:25.988466 30528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd4e8b7b426f41cb9b82b5c0016eb4c6: Sending DeleteTablet for 3 replicas of tablet 2c14272f7d324628a405c5ebd73eca49
I20260810 06:50:25.989590 31198 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c14272f7d324628a405c5ebd73eca49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:25 UTC) from {username='slave'} at 127.0.0.1:40906
I20260810 06:50:25.989581 30732 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c14272f7d324628a405c5ebd73eca49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:25 UTC) from {username='slave'} at 127.0.0.1:60084
I20260810 06:50:25.989600 30962 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c14272f7d324628a405c5ebd73eca49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:25 UTC) from {username='slave'} at 127.0.0.1:39970
I20260810 06:50:25.992635 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 30625
I20260810 06:50:26.002550 31399 tablet_replica.cc:330] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: stopping tablet replica
I20260810 06:50:26.003015 31399 raft_consensus.cc:2229] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:26.003564 31399 raft_consensus.cc:2258] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:26.004360 31401 tablet_replica.cc:330] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: stopping tablet replica
I20260810 06:50:26.005963 31340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:25.994204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60094 (local address 127.22.11.129:41531)
0810 06:50:25.994372 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:25.994377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:25.998144 (+ 3767us) server_negotiation.cc:407] Connection header received
0810 06:50:25.998319 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:25.998324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:25.998430 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:25.998533 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:26.005171 (+ 6638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:26.005177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:26.005181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:26.005203 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:26.005242 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:26.005259 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:26.005351 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:26.005595 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 06:50:26.005660 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:50:26.007151 30841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:26.007534 31401 raft_consensus.cc:2229] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:26.008248 31401 raft_consensus.cc:2258] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:26.008669 31399 ts_tablet_manager.cc:1905] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:26.010382 31401 ts_tablet_manager.cc:1905] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:26.016183 31399 ts_tablet_manager.cc:1918] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:26.016351 31399 log.cc:1196] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/2c14272f7d324628a405c5ebd73eca49
I20260810 06:50:26.016916 31399 ts_tablet_manager.cc:1939] T 2c14272f7d324628a405c5ebd73eca49 P f5457f8dcf4c427c9a0dace6ec7d87cc: Deleting consensus metadata
I20260810 06:50:26.017575 31401 ts_tablet_manager.cc:1918] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:26.017743 31401 log.cc:1196] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/2c14272f7d324628a405c5ebd73eca49
I20260810 06:50:26.018262 31401 ts_tablet_manager.cc:1939] T 2c14272f7d324628a405c5ebd73eca49 P 144142cea6344fdca0cdcd2b87211c2e: Deleting consensus metadata
I20260810 06:50:26.018854 30502 catalog_manager.cc:4841] TS f5457f8dcf4c427c9a0dace6ec7d87cc (127.22.11.131:36759): tablet 2c14272f7d324628a405c5ebd73eca49 (table test-table [id=2b730ececbcb4bbea02da4db6fa0d47f]) successfully deleted
I20260810 06:50:26.020165 30499 catalog_manager.cc:4841] TS 144142cea6344fdca0cdcd2b87211c2e (127.22.11.130:46519): tablet 2c14272f7d324628a405c5ebd73eca49 (table test-table [id=2b730ececbcb4bbea02da4db6fa0d47f]) successfully deleted
I20260810 06:50:26.142463 31397 tablet_replica.cc:330] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860: stopping tablet replica
I20260810 06:50:26.142900 31397 raft_consensus.cc:2229] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:26.143462 31397 raft_consensus.cc:2258] T 2c14272f7d324628a405c5ebd73eca49 P 3e47702bc75349808961ff2807baa860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:26.143841 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 30625
I20260810 06:50:26.175055 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 30856
W20260810 06:50:26.175065 30502 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.11.129:41531 (error 108)
I20260810 06:50:26.177316 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.176715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60104)
0810 06:50:26.176961 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:26.177155 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260810 06:50:26.177589 30502 catalog_manager.cc:4584] TS 3e47702bc75349808961ff2807baa860 (127.22.11.129:41531): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c14272f7d324628a405c5ebd73eca49: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
I20260810 06:50:26.178550 31341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.176215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.22.11.130:46519)
0810 06:50:26.176393 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:26.176399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:26.176421 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:50:26.176516 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:26.176522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:26.176592 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:26.176728 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:26.177709 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:26.177716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:26.177721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:26.177741 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:26.177781 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:26.177796 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:26.177942 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:26.178208 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 06:50:26.178271 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:50:26.179250 31081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:26.231070 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.230635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60108)
0810 06:50:26.230824 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:26.230914 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 06:50:26.232013 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.231677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60112)
0810 06:50:26.231819 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:26.231898 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:50:26.288745 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 30856
I20260810 06:50:26.292711 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.292175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60122)
0810 06:50:26.292395 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:26.292516 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 06:50:26.293788 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.293450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60138)
0810 06:50:26.293596 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:26.293670 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:50:26.310688 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 31091
I20260810 06:50:26.313169 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.311363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40916 (local address 127.22.11.131:36759)
0810 06:50:26.311514 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:26.311518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:26.311591 (+ 73us) server_negotiation.cc:407] Connection header received
0810 06:50:26.311755 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:26.311759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:26.311837 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:26.311911 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:26.312693 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:26.312697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:26.312700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:26.312712 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:26.312738 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:26.312749 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:26.312845 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:26.312900 (+ 55us) server_negotiation.cc:299] Negotiation successful
0810 06:50:26.312945 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:50:26.313732 31306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:26.374166 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.373674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60144)
0810 06:50:26.373877 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:26.373996 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 06:50:26.375293 31339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:26.374830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41531 (local address 127.0.0.1:60154)
0810 06:50:26.375056 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:26.375143 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260810 06:50:26.400775 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 31091
I20260810 06:50:26.426106 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 30467
I20260810 06:50:26.427235 30577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:26.531379 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 30467
2026-08-10T06:50:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2599 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 06:50:26.580788 22574 mini_oidc.cc:152] Starting JWKS server
I20260810 06:50:26.583575 22574 webserver.cc:460] Webserver started at https://127.0.0.1:46305/ using document root <none> and password file <none>
I20260810 06:50:26.583913 22574 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:50:26.584530 22574 webserver.cc:460] Webserver started at http://127.0.0.1:40175/ using document root <none> and password file <none>
2026-08-10T06:50:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:26Z Disabled control of system clock
I20260810 06:50:26.614158 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37449
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:39281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:37449
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46305/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/testchainca.cert with env {}
W20260810 06:50:26.930972 31424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:26.931253 31424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:26.931384 31424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:26.938488 31424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:26.938587 31424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:26.938647 31424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:26.938725 31424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:26.938778 31424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:50:26.938872 31424 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:50:26.938920 31424 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46305/jwks.json
I20260810 06:50:26.948752 31424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:39281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:37449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46305/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:26.950037 31424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:26.951347 31424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:26.962654 31439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:26.962715 31435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:26.962854 31437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:26.963708 31424 server_base.cc:1034] running on GCE node
I20260810 06:50:26.964443 31424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:26.965752 31424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:26.966962 31424 hybrid_clock.cc:648] HybridClock initialized: now 1786344626966866 us; error 88 us; skew 500 ppm
I20260810 06:50:26.967293 31424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:26.969508 31424 webserver.cc:460] Webserver started at http://127.22.11.190:39341/ using document root <none> and password file <none>
I20260810 06:50:26.970155 31424 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:26.970252 31424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:26.970554 31424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:26.973420 31424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "53b5eb2dbdb440adb72b42a8fee32ce9"
format_stamp: "Formatted at 2026-08-10 06:50:26 on dist-test-slave-f7th"
server_key: "eab1f8ee9637766059c4e4f83ea25230"
server_key_iv: "c77a97b94ae7328ffa893ffa9c66a206"
server_key_version: "encryptionkey@0"
I20260810 06:50:26.974169 31424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "53b5eb2dbdb440adb72b42a8fee32ce9"
format_stamp: "Formatted at 2026-08-10 06:50:26 on dist-test-slave-f7th"
server_key: "eab1f8ee9637766059c4e4f83ea25230"
server_key_iv: "c77a97b94ae7328ffa893ffa9c66a206"
server_key_version: "encryptionkey@0"
I20260810 06:50:26.978961 31424 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:50:26.982270 31454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:26.983577 31424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:26.983736 31424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "53b5eb2dbdb440adb72b42a8fee32ce9"
format_stamp: "Formatted at 2026-08-10 06:50:26 on dist-test-slave-f7th"
server_key: "eab1f8ee9637766059c4e4f83ea25230"
server_key_iv: "c77a97b94ae7328ffa893ffa9c66a206"
server_key_version: "encryptionkey@0"
I20260810 06:50:26.983918 31424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:26.999164 31424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:26.999890 31424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:27.000172 31424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:27.017524 31424 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:37449
I20260810 06:50:27.017550 31553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:37449 every 8 connection(s)
I20260810 06:50:27.018793 31424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:27.019625 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 31424
I20260810 06:50:27.019816 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:27.020589 22574 external_mini_cluster.cc:1428] Setting key c09bd2c4bc1d5c4a73eeced21488781a
I20260810 06:50:27.023623 31554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:27.033594 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:27.022502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59990 (local address 127.22.11.190:37449)
0810 06:50:27.023510 (+ 1008us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:27.023520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:27.023562 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:50:27.025257 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:27.025284 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:27.025694 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:27.026104 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:27.026666 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.027992 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.029393 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.030028 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.030200 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:27.031385 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:27.031393 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:27.031405 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:27.031965 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:27.032036 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:27.032063 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:27.032189 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:27.032406 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 06:50:27.032718 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":392,"threads_started":1}
I20260810 06:50:27.034255 31554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9: Bootstrap starting.
I20260810 06:50:27.037385 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:50:27.037583 31554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:27.038575 31554 log.cc:824] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:27.041119 31554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9: No bootstrap required, opened a new log
I20260810 06:50:27.047973 31554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37449 } }
I20260810 06:50:27.048286 31554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:27.048380 31554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b5eb2dbdb440adb72b42a8fee32ce9, State: Initialized, Role: FOLLOWER
I20260810 06:50:27.048929 31554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37449 } }
I20260810 06:50:27.049139 31554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:27.049259 31554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:27.049414 31554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:27.051450 31554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37449 } }
I20260810 06:50:27.051910 31554 leader_election.cc:302] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b5eb2dbdb440adb72b42a8fee32ce9; no voters:
I20260810 06:50:27.052418 31554 leader_election.cc:288] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:27.052557 31563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:27.053329 31563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [term 1 LEADER]: Becoming Leader. State: Replica: 53b5eb2dbdb440adb72b42a8fee32ce9, State: Running, Role: LEADER
I20260810 06:50:27.053867 31563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37449 } }
I20260810 06:50:27.054409 31554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:27.057102 31565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37449 } } }
I20260810 06:50:27.057181 31566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b5eb2dbdb440adb72b42a8fee32ce9. Latest consensus state: current_term: 1 leader_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b5eb2dbdb440adb72b42a8fee32ce9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37449 } } }
I20260810 06:50:27.058022 31565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:27.058035 31566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:27.059213 31579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:27.063781 31579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:27.071695 31579 catalog_manager.cc:1347] Generated new cluster ID: 8bcac2f8f1af4ef6a5ec6515a6b80b83
I20260810 06:50:27.071794 31579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:27.079483 31579 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:27.080721 31579 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:27.090947 31579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b5eb2dbdb440adb72b42a8fee32ce9: Generated new TSK 0
I20260810 06:50:27.091686 31579 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:50:27.139285 31606 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 06:50:27.139496 31606 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:50:27.142030 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:27.118107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.22.11.190:37449)
0810 06:50:27.118310 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:27.118316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:27.118747 (+ 431us) server_negotiation.cc:407] Connection header received
0810 06:50:27.119087 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:27.119092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:27.119186 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:27.119341 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:50:27.119842 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.121006 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.122472 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.123148 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.123272 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:27.123773 (+ 501us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:50:27.141339 (+ 17566us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46305/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:50:27.141710 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46305/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 06:50:27.142156 31558 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59996: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46305/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 06:50:27.143471 31604 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:27.117969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:37449 (local address 127.0.0.1:59996)
0810 06:50:27.118649 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:27.118670 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:27.118820 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:27.119366 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:27.119370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:27.119380 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:50:27.119697 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:27.119706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.121158 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.121162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:27.122315 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:27.122328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.123303 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.123307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:27.123445 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:27.123499 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:50:27.141616 (+ 18117us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46305/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 06:50:27.141870 (+ 254us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.11.190:37449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46305/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":361,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.22.11.190:37449: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46305/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 06:50:27.145718 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 31424
I20260810 06:50:27.146873 31544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:27.238442 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 31424
2026-08-10T06:50:27Z chronyd exiting
I20260810 06:50:27.293084 22574 test_util.cc:175] -----------------------------------------------
I20260810 06:50:27.293220 22574 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786344580063516-22574-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (717 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 06:50:27.296893 22574 mini_oidc.cc:152] Starting JWKS server
I20260810 06:50:27.298669 22574 webserver.cc:460] Webserver started at https://127.0.0.1:35703/ using document root <none> and password file <none>
I20260810 06:50:27.298875 22574 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:50:27.299516 22574 webserver.cc:460] Webserver started at http://127.0.0.1:36073/ using document root <none> and password file <none>
2026-08-10T06:50:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:27Z Disabled control of system clock
I20260810 06:50:27.330142 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43947
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:43947
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35703/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/testchainca.cert with env {}
W20260810 06:50:27.645015 31617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:27.645303 31617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:27.645444 31617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:27.652539 31617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:27.652665 31617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:27.652733 31617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:27.652815 31617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:27.652881 31617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:50:27.652968 31617 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:50:27.653020 31617 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35703/jwks.json
I20260810 06:50:27.662631 31617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:43947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35703/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:27.663921 31617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:27.665203 31617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:27.676409 31625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:27.676443 31626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:27.676308 31629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:27.677234 31617 server_base.cc:1034] running on GCE node
I20260810 06:50:27.677968 31617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:27.679291 31617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:27.680510 31617 hybrid_clock.cc:648] HybridClock initialized: now 1786344627680454 us; error 46 us; skew 500 ppm
I20260810 06:50:27.680852 31617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:27.683001 31617 webserver.cc:460] Webserver started at http://127.22.11.190:46655/ using document root <none> and password file <none>
I20260810 06:50:27.683665 31617 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:27.683772 31617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:27.684082 31617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:27.686975 31617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "0b2926ced43042a9b26c943e6edf91b5"
format_stamp: "Formatted at 2026-08-10 06:50:27 on dist-test-slave-f7th"
server_key: "0ee484a4f4910ef1735d7c008d3a75e6"
server_key_iv: "fb19d2620c10c98f6169a0eaa7710e1a"
server_key_version: "encryptionkey@0"
I20260810 06:50:27.687739 31617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "0b2926ced43042a9b26c943e6edf91b5"
format_stamp: "Formatted at 2026-08-10 06:50:27 on dist-test-slave-f7th"
server_key: "0ee484a4f4910ef1735d7c008d3a75e6"
server_key_iv: "fb19d2620c10c98f6169a0eaa7710e1a"
server_key_version: "encryptionkey@0"
I20260810 06:50:27.692471 31617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 06:50:27.695659 31641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:27.696862 31617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:50:27.697021 31617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "0b2926ced43042a9b26c943e6edf91b5"
format_stamp: "Formatted at 2026-08-10 06:50:27 on dist-test-slave-f7th"
server_key: "0ee484a4f4910ef1735d7c008d3a75e6"
server_key_iv: "fb19d2620c10c98f6169a0eaa7710e1a"
server_key_version: "encryptionkey@0"
I20260810 06:50:27.697220 31617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:27.717303 31617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:27.718053 31617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:27.718333 31617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:27.735153 31617 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:43947
I20260810 06:50:27.735183 31740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:43947 every 8 connection(s)
I20260810 06:50:27.736400 31617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:27.740011 31742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:27.746614 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 31617
I20260810 06:50:27.746798 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:27.747623 22574 external_mini_cluster.cc:1428] Setting key 24ceae8edebb24db5977562aa7105fcc
I20260810 06:50:27.750478 31742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5: Bootstrap starting.
I20260810 06:50:27.753676 31742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:27.754674 31742 log.cc:824] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:27.757571 31742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5: No bootstrap required, opened a new log
I20260810 06:50:27.760784 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:27.749435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.22.11.190:43947)
0810 06:50:27.750353 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:27.750363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:27.750402 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:50:27.752035 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:27.752063 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:27.752466 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:27.752788 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:27.753317 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.754588 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.756098 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.756786 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.756907 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:27.758125 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:27.758136 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:27.758153 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:27.758830 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:27.758925 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:27.758957 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:27.759077 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:27.759232 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:50:27.759547 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":324,"threads_started":1}
I20260810 06:50:27.764693 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:50:27.767359 31742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2926ced43042a9b26c943e6edf91b5" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43947 } }
I20260810 06:50:27.767746 31742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:27.767849 31742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b2926ced43042a9b26c943e6edf91b5, State: Initialized, Role: FOLLOWER
I20260810 06:50:27.768481 31742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2926ced43042a9b26c943e6edf91b5" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43947 } }
I20260810 06:50:27.768764 31742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:27.768896 31742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:27.769052 31742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:27.771101 31742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2926ced43042a9b26c943e6edf91b5" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43947 } }
I20260810 06:50:27.771649 31742 leader_election.cc:302] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b2926ced43042a9b26c943e6edf91b5; no voters:
I20260810 06:50:27.772168 31742 leader_election.cc:288] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:27.772292 31749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:27.773001 31749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [term 1 LEADER]: Becoming Leader. State: Replica: 0b2926ced43042a9b26c943e6edf91b5, State: Running, Role: LEADER
I20260810 06:50:27.773487 31749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2926ced43042a9b26c943e6edf91b5" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43947 } }
I20260810 06:50:27.774044 31742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:27.776204 31752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b2926ced43042a9b26c943e6edf91b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2926ced43042a9b26c943e6edf91b5" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43947 } } }
I20260810 06:50:27.776173 31754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b2926ced43042a9b26c943e6edf91b5. Latest consensus state: current_term: 1 leader_uuid: "0b2926ced43042a9b26c943e6edf91b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b2926ced43042a9b26c943e6edf91b5" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43947 } } }
I20260810 06:50:27.776562 31754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:27.776562 31752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:27.778463 31761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:27.783388 31761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:27.791998 31761 catalog_manager.cc:1347] Generated new cluster ID: 3a14d229b14a4b9b90c5cb1a1664a7fa
I20260810 06:50:27.792109 31761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:27.816598 31761 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:27.818395 31761 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:27.830654 31761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b2926ced43042a9b26c943e6edf91b5: Generated new TSK 0
I20260810 06:50:27.831512 31761 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 06:50:27.868671 31786 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 06:50:27.868806 31786 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:50:27.869678 31745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:27.842838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.22.11.190:43947)
0810 06:50:27.843019 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:27.843025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:27.843519 (+ 494us) server_negotiation.cc:407] Connection header received
0810 06:50:27.843753 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:27.843757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:27.843822 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:27.843944 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:50:27.844447 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.845207 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.847176 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.847615 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.847706 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:27.848214 (+ 508us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:50:27.869168 (+ 20954us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35703/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:50:27.869466 (+ 298us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35703/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":45}
W20260810 06:50:27.869794 31745 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52296: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35703/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 06:50:27.870070 31783 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:27.842614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43947 (local address 127.0.0.1:52296)
0810 06:50:27.843413 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:27.843433 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:27.843582 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:27.843970 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:27.843975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:27.843989 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:50:27.844289 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:27.844301 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.845347 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.845353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:27.847048 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:27.847063 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:27.847760 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:27.847767 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:27.847949 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:27.848004 (+ 55us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:50:27.869394 (+ 21390us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35703/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 06:50:27.869665 (+ 271us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.11.190:43947: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35703/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":485,"threads_started":1}
I20260810 06:50:27.871263 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 31617
I20260810 06:50:27.872459 31734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:27.964293 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 31617
2026-08-10T06:50:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (731 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 06:50:28.028164 22574 mini_oidc.cc:152] Starting JWKS server
I20260810 06:50:28.030001 22574 webserver.cc:460] Webserver started at https://127.0.0.1:44861/ using document root <none> and password file <none>
I20260810 06:50:28.030212 22574 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 06:50:28.030777 22574 webserver.cc:460] Webserver started at http://127.0.0.1:39239/ using document root <none> and password file <none>
2026-08-10T06:50:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:28Z Disabled control of system clock
I20260810 06:50:28.061417 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39031
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:39065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:39031
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44861/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:50:28.375258 31799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:28.375524 31799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:28.375661 31799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:28.382642 31799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:28.382740 31799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:28.382808 31799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:28.382896 31799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:28.382959 31799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:50:28.383044 31799 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:50:28.383096 31799 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44861/jwks.json
I20260810 06:50:28.392304 31799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:39065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:39031
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44861/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:28.393563 31799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:28.395282 31799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:28.406602 31804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:28.406699 31806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:28.406930 31808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:28.407619 31799 server_base.cc:1034] running on GCE node
I20260810 06:50:28.408326 31799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:28.409700 31799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:28.410912 31799 hybrid_clock.cc:648] HybridClock initialized: now 1786344628410867 us; error 36 us; skew 500 ppm
I20260810 06:50:28.411242 31799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:28.417135 31799 webserver.cc:460] Webserver started at http://127.22.11.190:38445/ using document root <none> and password file <none>
I20260810 06:50:28.417799 31799 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:28.417900 31799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:28.418216 31799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:28.421065 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "81e77db8133c4daa829916aedd476202"
format_stamp: "Formatted at 2026-08-10 06:50:28 on dist-test-slave-f7th"
server_key: "60287b6bf6d3e943b9b4ad16cd1a3ab5"
server_key_iv: "971dfe07bf23ddcfc976608ffef2d901"
server_key_version: "encryptionkey@0"
I20260810 06:50:28.421828 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "81e77db8133c4daa829916aedd476202"
format_stamp: "Formatted at 2026-08-10 06:50:28 on dist-test-slave-f7th"
server_key: "60287b6bf6d3e943b9b4ad16cd1a3ab5"
server_key_iv: "971dfe07bf23ddcfc976608ffef2d901"
server_key_version: "encryptionkey@0"
I20260810 06:50:28.426762 31799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 06:50:28.430006 31822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:28.431314 31799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:28.431475 31799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "81e77db8133c4daa829916aedd476202"
format_stamp: "Formatted at 2026-08-10 06:50:28 on dist-test-slave-f7th"
server_key: "60287b6bf6d3e943b9b4ad16cd1a3ab5"
server_key_iv: "971dfe07bf23ddcfc976608ffef2d901"
server_key_version: "encryptionkey@0"
I20260810 06:50:28.431681 31799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:28.452086 31799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:28.452879 31799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:28.453169 31799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:28.469600 31799 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:39031
I20260810 06:50:28.469636 31907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:39031 every 8 connection(s)
I20260810 06:50:28.470821 31799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:28.474309 31913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:28.476946 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 31799
I20260810 06:50:28.477169 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:28.477942 22574 external_mini_cluster.cc:1428] Setting key 4a025141dcf9c369939e873ce730109f
I20260810 06:50:28.486140 31913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202: Bootstrap starting.
I20260810 06:50:28.489414 31913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:28.490483 31913 log.cc:824] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:28.490958 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:28.479766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39484 (local address 127.22.11.190:39031)
0810 06:50:28.480707 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:28.480717 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:28.480758 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:50:28.482339 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:28.482361 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:28.482799 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:28.483113 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:28.483695 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:28.485051 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:28.486799 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:28.487456 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:28.487665 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:28.488777 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:28.488784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:28.488799 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:28.489297 (+ 498us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:28.489388 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:28.489411 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:28.489517 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:28.489748 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 06:50:28.490033 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":333,"threads_started":1}
I20260810 06:50:28.493495 31913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202: No bootstrap required, opened a new log
I20260810 06:50:28.494669 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:50:28.501317 31913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e77db8133c4daa829916aedd476202" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39031 } }
I20260810 06:50:28.501629 31913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:28.501724 31913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81e77db8133c4daa829916aedd476202, State: Initialized, Role: FOLLOWER
I20260810 06:50:28.502228 31913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e77db8133c4daa829916aedd476202" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39031 } }
I20260810 06:50:28.502427 31913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:28.502554 31913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:28.502709 31913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:28.504868 31913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e77db8133c4daa829916aedd476202" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39031 } }
I20260810 06:50:28.505316 31913 leader_election.cc:302] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81e77db8133c4daa829916aedd476202; no voters:
I20260810 06:50:28.505957 31913 leader_election.cc:288] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:28.506078 31919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:28.506987 31919 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [term 1 LEADER]: Becoming Leader. State: Replica: 81e77db8133c4daa829916aedd476202, State: Running, Role: LEADER
I20260810 06:50:28.507455 31919 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e77db8133c4daa829916aedd476202" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39031 } }
I20260810 06:50:28.507778 31913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:28.510070 31921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81e77db8133c4daa829916aedd476202. Latest consensus state: current_term: 1 leader_uuid: "81e77db8133c4daa829916aedd476202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e77db8133c4daa829916aedd476202" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39031 } } }
I20260810 06:50:28.510176 31920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81e77db8133c4daa829916aedd476202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81e77db8133c4daa829916aedd476202" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39031 } } }
I20260810 06:50:28.510484 31921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:28.510524 31920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:28.511400 31933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:28.516729 31933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:28.524819 31933 catalog_manager.cc:1347] Generated new cluster ID: 7121855cb98e4a348ad47e1c6049d6d8
I20260810 06:50:28.524914 31933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:28.537207 31933 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:28.538502 31933 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:28.553920 31933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81e77db8133c4daa829916aedd476202: Generated new TSK 0
I20260810 06:50:28.554919 31933 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:50:28.575413 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:28.567841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39490 (local address 127.22.11.190:39031)
0810 06:50:28.568062 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:28.568068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:28.568407 (+ 339us) server_negotiation.cc:407] Connection header received
0810 06:50:28.568652 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:28.568658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:28.568737 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:28.568852 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:50:28.569385 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:28.570107 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:28.572034 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:28.572447 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:28.572529 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:28.573118 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:50:28.574814 (+ 1696us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:50:28.575135 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39490: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 06:50:28.575532 31917 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39490: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 06:50:28.575667 31957 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:28.567728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39031 (local address 127.0.0.1:39490)
0810 06:50:28.568308 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:28.568327 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:28.568469 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:28.568891 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:28.568896 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:28.568909 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:50:28.569244 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:28.569257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:28.570232 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:28.570238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:28.571897 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:28.571913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:28.572644 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:28.572651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:28.572840 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:28.572886 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:50:28.575004 (+ 2118us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 06:50:28.575234 (+ 230us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.11.190:39031: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44861/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":313,"threads_started":1}
I20260810 06:50:28.576884 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 31799
I20260810 06:50:28.577834 31899 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 06:50:28.587397 31958 webserver.cc:533] Webserver: sslize failed:
E20260810 06:50:28.587500 31958 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 06:50:28.668888 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 31799
2026-08-10T06:50:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (702 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T06:50:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:28Z Disabled control of system clock
I20260810 06:50:28.760035 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:35443
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:45823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:35443
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/testchainca.cert with env {}
W20260810 06:50:29.086717 31965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:29.087059 31965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:29.087198 31965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:29.098354 31965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:29.098495 31965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:29.098564 31965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:29.098649 31965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:29.098729 31965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 06:50:29.098822 31965 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 06:50:29.098884 31965 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 06:50:29.112901 31965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:35443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:35443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:29.114179 31965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:29.115449 31965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:29.126613 31974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:29.126695 31973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:29.126700 31977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:29.127599 31965 server_base.cc:1034] running on GCE node
I20260810 06:50:29.128394 31965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:29.129778 31965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:29.130996 31965 hybrid_clock.cc:648] HybridClock initialized: now 1786344629130944 us; error 42 us; skew 500 ppm
I20260810 06:50:29.131333 31965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:50:29.133502 31965 webserver.cc:460] Webserver started at http://127.22.11.190:46075/ using document root <none> and password file <none>
I20260810 06:50:29.134155 31965 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:29.134259 31965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:29.134543 31965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:29.137385 31965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "b8607932d91c4174b54ab60f181342d1"
format_stamp: "Formatted at 2026-08-10 06:50:29 on dist-test-slave-f7th"
server_key: "abae5c92b60cf3f2f695db9f019fb713"
server_key_iv: "461997e53a68fd778e821a91727d058c"
server_key_version: "encryptionkey@0"
I20260810 06:50:29.138154 31965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "b8607932d91c4174b54ab60f181342d1"
format_stamp: "Formatted at 2026-08-10 06:50:29 on dist-test-slave-f7th"
server_key: "abae5c92b60cf3f2f695db9f019fb713"
server_key_iv: "461997e53a68fd778e821a91727d058c"
server_key_version: "encryptionkey@0"
I20260810 06:50:29.142975 31965 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:50:29.146282 31995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:29.147562 31965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:29.147728 31965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "b8607932d91c4174b54ab60f181342d1"
format_stamp: "Formatted at 2026-08-10 06:50:29 on dist-test-slave-f7th"
server_key: "abae5c92b60cf3f2f695db9f019fb713"
server_key_iv: "461997e53a68fd778e821a91727d058c"
server_key_version: "encryptionkey@0"
I20260810 06:50:29.147926 31965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:29.164187 31965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:29.164986 31965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:29.165244 31965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:29.182003 31965 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:35443
I20260810 06:50:29.182025 32083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:35443 every 8 connection(s)
I20260810 06:50:29.183259 31965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:29.186852 32084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:29.188419 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 31965
I20260810 06:50:29.188632 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:29.189277 22574 external_mini_cluster.cc:1428] Setting key 818476b89c26d9d8dcbff1b52bb59d39
I20260810 06:50:29.199452 32084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1: Bootstrap starting.
I20260810 06:50:29.203229 32084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:29.204098 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:29.191064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.22.11.190:35443)
0810 06:50:29.192127 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:29.192140 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:29.192189 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:50:29.194478 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:29.194515 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:29.195038 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:29.195443 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:29.195973 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:29.197929 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:29.199140 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:29.200135 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:29.200316 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:29.201406 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:29.201417 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:29.201433 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:50:29.202098 (+ 665us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:29.202210 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:50:29.202238 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:29.202355 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:29.202548 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 06:50:29.202839 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":319,"threads_started":1}
I20260810 06:50:29.204509 32084 log.cc:824] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:29.207551 32084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1: No bootstrap required, opened a new log
I20260810 06:50:29.207949 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:50:29.215246 32084 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8607932d91c4174b54ab60f181342d1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35443 } }
I20260810 06:50:29.215602 32084 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:29.215692 32084 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8607932d91c4174b54ab60f181342d1, State: Initialized, Role: FOLLOWER
I20260810 06:50:29.216220 32084 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8607932d91c4174b54ab60f181342d1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35443 } }
I20260810 06:50:29.216434 32084 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:29.216586 32084 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:29.216791 32084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:29.218937 32084 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8607932d91c4174b54ab60f181342d1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35443 } }
I20260810 06:50:29.219369 32084 leader_election.cc:302] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8607932d91c4174b54ab60f181342d1; no voters:
I20260810 06:50:29.219856 32084 leader_election.cc:288] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:29.219986 32095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:29.220650 32095 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [term 1 LEADER]: Becoming Leader. State: Replica: b8607932d91c4174b54ab60f181342d1, State: Running, Role: LEADER
I20260810 06:50:29.221113 32095 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8607932d91c4174b54ab60f181342d1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35443 } }
I20260810 06:50:29.221777 32084 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:29.223785 32098 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8607932d91c4174b54ab60f181342d1. Latest consensus state: current_term: 1 leader_uuid: "b8607932d91c4174b54ab60f181342d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8607932d91c4174b54ab60f181342d1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35443 } } }
I20260810 06:50:29.224195 32098 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:29.223996 32097 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8607932d91c4174b54ab60f181342d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8607932d91c4174b54ab60f181342d1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35443 } } }
I20260810 06:50:29.224308 32097 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:29.225554 32106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:29.230618 32106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:29.238888 32106 catalog_manager.cc:1347] Generated new cluster ID: e89f5a1e82034d1381b5680a2fdbe078
I20260810 06:50:29.238986 32106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:29.254261 32106 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:29.255936 32106 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:29.263715 32106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8607932d91c4174b54ab60f181342d1: Generated new TSK 0
I20260810 06:50:29.264678 32106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:50:29.311213 32089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:29.282300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.22.11.190:35443)
0810 06:50:29.282483 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:29.282490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:29.282884 (+ 394us) server_negotiation.cc:407] Connection header received
0810 06:50:29.283116 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:29.283121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:29.283233 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:29.283392 (+ 159us) server_negotiation.cc:226] Negotiated authn=JWT
0810 06:50:29.283896 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:29.285031 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:29.286397 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:29.287068 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:29.287194 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:29.287621 (+ 427us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 06:50:29.310503 (+ 22882us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:50:29.310874 (+ 371us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34300: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 06:50:29.311256 32126 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:29.282141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:35443 (local address 127.0.0.1:34300)
0810 06:50:29.282794 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:29.282810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:29.282921 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:29.283406 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:29.283411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:29.283423 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0810 06:50:29.283730 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:29.283741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:29.285175 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:29.285178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:29.286263 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:29.286274 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:29.287234 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:29.287239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:29.287354 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:29.287386 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 06:50:29.310734 (+ 23348us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 06:50:29.310955 (+ 221us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.22.11.190:35443: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":382,"threads_started":1}
W20260810 06:50:29.311357 32089 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34300: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260810 06:50:29.312794 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 31965
I20260810 06:50:29.313874 32075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:29.404589 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 31965
2026-08-10T06:50:29Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (714 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (408 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (399 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:30 dist-test-slave-f7th krb5kdc[32151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:30 dist-test-slave-f7th krb5kdc[32151](info): set up 2 sockets
Aug 10 06:50:30 dist-test-slave-f7th krb5kdc[32151](info): commencing operation
krb5kdc: starting...
W20260810 06:50:31.442095 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:31 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:31.649853 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:41043
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:41043
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:31.966777 32176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:31.967087 32176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:31.967233 32176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:31.974267 32176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:31.974359 32176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:31.974433 32176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:31.974519 32176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:31.974576 32176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:31.983933 32176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:41043
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:41043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:31.985155 32176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:31.986404 32176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:31.997566 32183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:31.997824 32182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:31.997817 32190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:31.998668 32176 server_base.cc:1034] running on GCE node
I20260810 06:50:31.999415 32176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:32.000669 32176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:32.001865 32176 hybrid_clock.cc:648] HybridClock initialized: now 1786344632001811 us; error 46 us; skew 500 ppm
I20260810 06:50:32.002168 32176 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:32 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344632, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:32.005060 32176 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:32.006347 32176 webserver.cc:460] Webserver started at http://127.22.11.190:33941/ using document root <none> and password file <none>
I20260810 06:50:32.006963 32176 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:32.007064 32176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:32.007315 32176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:32.010216 32176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "b16aee4a40c642bea1c6f53724adbb81"
format_stamp: "Formatted at 2026-08-10 06:50:32 on dist-test-slave-f7th"
server_key: "a46a7c10bd75bb3ee7e564dffe7210aa"
server_key_iv: "3126a89113ed40b955fe2a4b05cdd235"
server_key_version: "encryptionkey@0"
I20260810 06:50:32.010924 32176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "b16aee4a40c642bea1c6f53724adbb81"
format_stamp: "Formatted at 2026-08-10 06:50:32 on dist-test-slave-f7th"
server_key: "a46a7c10bd75bb3ee7e564dffe7210aa"
server_key_iv: "3126a89113ed40b955fe2a4b05cdd235"
server_key_version: "encryptionkey@0"
I20260810 06:50:32.015681 32176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260810 06:50:32.018925 32205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:32.020134 32176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:50:32.020264 32176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "b16aee4a40c642bea1c6f53724adbb81"
format_stamp: "Formatted at 2026-08-10 06:50:32 on dist-test-slave-f7th"
server_key: "a46a7c10bd75bb3ee7e564dffe7210aa"
server_key_iv: "3126a89113ed40b955fe2a4b05cdd235"
server_key_version: "encryptionkey@0"
I20260810 06:50:32.020438 32176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:32.037917 32176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:32.041484 32176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:32.041697 32176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:32.058370 32176 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:41043
I20260810 06:50:32.058436 32298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:41043 every 8 connection(s)
I20260810 06:50:32.059593 32176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:32.063150 32301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:32.069074 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 32176
I20260810 06:50:32.069298 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:32.070123 22574 external_mini_cluster.cc:1428] Setting key 8e40563a975f9114cdcf4ef5d4583a80
I20260810 06:50:32.074141 32301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81: Bootstrap starting.
I20260810 06:50:32.077116 32301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:32.078130 32301 log.cc:824] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:32.080796 32301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81: No bootstrap required, opened a new log
Aug 10 06:50:32 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:32.087936 32301 raft_consensus.cc:348] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16aee4a40c642bea1c6f53724adbb81" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41043 } }
I20260810 06:50:32.088291 32301 raft_consensus.cc:374] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:32.088389 32301 raft_consensus.cc:729] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16aee4a40c642bea1c6f53724adbb81, State: Initialized, Role: FOLLOWER
I20260810 06:50:32.088986 32301 consensus_queue.cc:260] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16aee4a40c642bea1c6f53724adbb81" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41043 } }
I20260810 06:50:32.089246 32301 raft_consensus.cc:388] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:32.089390 32301 raft_consensus.cc:482] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:32.089561 32301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:32.091126 32304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:32.072296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.22.11.190:41043)
0810 06:50:32.073050 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:32.073061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:32.073099 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:50:32.074799 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:32.074821 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:32.075171 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:32.075534 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:32.076953 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:32.078264 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:32.079758 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:32.080389 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:32.080549 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:32.084204 (+ 3655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:32.084212 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:32.084228 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:32.084274 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:32.087272 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:32.087911 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:32.087918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:32.087927 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:32.088087 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:32.088471 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:32.088476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:32.088479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:32.089047 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:32.089327 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:32.089768 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 06:50:32.090183 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":338,"threads_started":1}
I20260810 06:50:32.092196 32301 raft_consensus.cc:504] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16aee4a40c642bea1c6f53724adbb81" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41043 } }
I20260810 06:50:32.092760 32301 leader_election.cc:302] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b16aee4a40c642bea1c6f53724adbb81; no voters:
I20260810 06:50:32.093418 32301 leader_election.cc:288] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:32.093541 32307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:32.094374 32307 raft_consensus.cc:686] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [term 1 LEADER]: Becoming Leader. State: Replica: b16aee4a40c642bea1c6f53724adbb81, State: Running, Role: LEADER
I20260810 06:50:32.094935 32307 consensus_queue.cc:237] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16aee4a40c642bea1c6f53724adbb81" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41043 } }
I20260810 06:50:32.096908 32307 sys_catalog.cc:455] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b16aee4a40c642bea1c6f53724adbb81. Latest consensus state: current_term: 1 leader_uuid: "b16aee4a40c642bea1c6f53724adbb81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16aee4a40c642bea1c6f53724adbb81" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41043 } } }
I20260810 06:50:32.097261 32307 sys_catalog.cc:458] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:32.097740 32311 sys_catalog.cc:455] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b16aee4a40c642bea1c6f53724adbb81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b16aee4a40c642bea1c6f53724adbb81" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41043 } } }
I20260810 06:50:32.098109 32311 sys_catalog.cc:458] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:32.101691 32301 sys_catalog.cc:564] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:32.104321 32315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:32.109462 32315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:32.117928 32315 catalog_manager.cc:1347] Generated new cluster ID: eaec8b56ae4f4bdd85cc98a46d197cdb
I20260810 06:50:32.118042 32315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:32.135309 32315 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:32.136459 32315 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:32.147289 32315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b16aee4a40c642bea1c6f53724adbb81: Generated new TSK 0
I20260810 06:50:32.148018 32315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:50:32.211227 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:41043
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:32.554494 32339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:32.554740 32339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:32.554883 32339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:32.561904 32339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:32.562009 32339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:32.562163 32339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:32.571446 32339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:41043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:32.572643 32339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:32.573947 32339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:32.587458 32348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:32.587394 32352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:32.587363 32355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:32.588274 32339 server_base.cc:1034] running on GCE node
I20260810 06:50:32.588891 32339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:32.589783 32339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:32.591006 32339 hybrid_clock.cc:648] HybridClock initialized: now 1786344632590936 us; error 51 us; skew 500 ppm
I20260810 06:50:32.591296 32339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:32 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344632, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:32.594503 32339 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:50:32.595834 32339 webserver.cc:460] Webserver started at http://127.22.11.129:46423/ using document root <none> and password file <none>
I20260810 06:50:32.596453 32339 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:32.596555 32339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:32.596858 32339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:32.599736 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "ed48222637e04ef5ba23b8206ee18467"
format_stamp: "Formatted at 2026-08-10 06:50:32 on dist-test-slave-f7th"
server_key: "0a6fb337967aee61799155648d7b8a21"
server_key_iv: "d4c81bbbbc356d58838a970334e2948c"
server_key_version: "encryptionkey@0"
I20260810 06:50:32.600466 32339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "ed48222637e04ef5ba23b8206ee18467"
format_stamp: "Formatted at 2026-08-10 06:50:32 on dist-test-slave-f7th"
server_key: "0a6fb337967aee61799155648d7b8a21"
server_key_iv: "d4c81bbbbc356d58838a970334e2948c"
server_key_version: "encryptionkey@0"
I20260810 06:50:32.605437 32339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:50:32.608714 32370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:32.609913 32339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 06:50:32.610067 32339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "ed48222637e04ef5ba23b8206ee18467"
format_stamp: "Formatted at 2026-08-10 06:50:32 on dist-test-slave-f7th"
server_key: "0a6fb337967aee61799155648d7b8a21"
server_key_iv: "d4c81bbbbc356d58838a970334e2948c"
server_key_version: "encryptionkey@0"
I20260810 06:50:32.610248 32339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:32.636191 32339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:32.639976 32339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:32.640241 32339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:32.641196 32339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:32.642489 32339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:32.642571 32339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:32.642691 32339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:32.642745 32339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:32.672606 32339 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:35085
I20260810 06:50:32.672698 32563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:35085 every 8 connection(s)
I20260810 06:50:32.673906 32339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:32.676093 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 32339
I20260810 06:50:32.676280 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:32.677088 22574 external_mini_cluster.cc:1428] Setting key 2045991dbc50c44b53bb7f4ea751a00b
Aug 10 06:50:32 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344632, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:32.695482 32569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:32.676436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:38517 (local address 127.22.11.190:41043)
0810 06:50:32.676963 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:32.676969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:32.677975 (+ 1006us) server_negotiation.cc:407] Connection header received
0810 06:50:32.680180 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:32.680184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:32.680247 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:32.680337 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:32.682282 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:32.683027 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:32.685070 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:32.685481 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:32.685587 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:32.689366 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:32.689373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:32.689377 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:32.689417 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:32.692411 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:32.693171 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:32.693176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:32.693179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:32.693322 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:32.693700 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:32.693704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:32.693707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:32.694097 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:32.694244 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:32.694892 (+ 648us) server_negotiation.cc:299] Negotiation successful
0810 06:50:32.695118 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":263,"threads_started":1}
I20260810 06:50:32.696101 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:32.677031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:41043 (local address 127.22.11.129:38517)
0810 06:50:32.677819 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:32.677867 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:32.679903 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:32.680586 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:32.680597 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:32.681338 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:32.682100 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:32.682125 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:32.683189 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:32.683194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:32.684896 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:32.684916 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:32.685697 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:32.685704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:32.685962 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:32.686636 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:32.686661 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:32.689139 (+ 2478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:32.692584 (+ 3445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:32.692593 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:32.692609 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:32.693041 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:32.693419 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:32.693422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:32.693424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:32.693593 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:32.694216 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:32.694221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:32.694502 (+ 281us) client_negotiation.cc:769] Sending connection context
0810 06:50:32.694724 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 06:50:32.695078 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":330,"threads_started":1}
I20260810 06:50:32.697548 32565 heartbeater.cc:347] Connected to a master server at 127.22.11.190:41043
I20260810 06:50:32.697916 32565 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:32.698740 32565 heartbeater.cc:511] Master 127.22.11.190:41043 requested a full tablet report, sending...
I20260810 06:50:32.700950 32241 ts_manager.cc:194] Registered new tserver with Master: ed48222637e04ef5ba23b8206ee18467 (127.22.11.129:35085)
I20260810 06:50:32.702474 32241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:38517
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:50:32.729229 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:41043
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:33.052325 32576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:33.052634 32576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:33.052794 32576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:33.060137 32576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:33.060233 32576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:33.060370 32576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:33.069753 32576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:41043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:33.070889 32576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:33.072151 32576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:33.084648 32592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:33.084703 32586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:33.084712 32589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:33.085427 32576 server_base.cc:1034] running on GCE node
I20260810 06:50:33.086041 32576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:33.086920 32576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:33.088107 32576 hybrid_clock.cc:648] HybridClock initialized: now 1786344633088066 us; error 33 us; skew 500 ppm
I20260810 06:50:33.088397 32576 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.091418 32576 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:50:33.092769 32576 webserver.cc:460] Webserver started at http://127.22.11.130:36437/ using document root <none> and password file <none>
I20260810 06:50:33.093464 32576 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:33.093578 32576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:33.093865 32576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:33.096769 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "72dc633a01b74bebba881fbdf4e61c5d"
format_stamp: "Formatted at 2026-08-10 06:50:33 on dist-test-slave-f7th"
server_key: "24352886b20c72b9873e1069bb06b8e9"
server_key_iv: "10c016940452204eabb4216f62918c81"
server_key_version: "encryptionkey@0"
I20260810 06:50:33.097517 32576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "72dc633a01b74bebba881fbdf4e61c5d"
format_stamp: "Formatted at 2026-08-10 06:50:33 on dist-test-slave-f7th"
server_key: "24352886b20c72b9873e1069bb06b8e9"
server_key_iv: "10c016940452204eabb4216f62918c81"
server_key_version: "encryptionkey@0"
I20260810 06:50:33.102284 32576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:50:33.105568 32607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:33.106781 32576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:50:33.106943 32576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "72dc633a01b74bebba881fbdf4e61c5d"
format_stamp: "Formatted at 2026-08-10 06:50:33 on dist-test-slave-f7th"
server_key: "24352886b20c72b9873e1069bb06b8e9"
server_key_iv: "10c016940452204eabb4216f62918c81"
server_key_version: "encryptionkey@0"
I20260810 06:50:33.107123 32576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:33.120498 32576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:33.123935 32576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:33.124183 32576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:33.125113 32576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:33.126363 32576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:33.126439 32576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:33.126556 32576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:33.126616 32576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:33.156008 32576 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35049
I20260810 06:50:33.156049 330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35049 every 8 connection(s)
I20260810 06:50:33.157279 32576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:33.166873 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 32576
I20260810 06:50:33.167057 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:33.167618 22574 external_mini_cluster.cc:1428] Setting key 0e1f02ac98265893ad143a43912c92c3
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:33.176409 32569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.159677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:38643 (local address 127.22.11.190:41043)
0810 06:50:33.159847 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.159852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.161008 (+ 1156us) server_negotiation.cc:407] Connection header received
0810 06:50:33.162696 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.162701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.162769 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.162853 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.164213 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.164958 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.166377 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.166792 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.166894 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.170543 (+ 3649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:33.170549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:33.170553 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:33.170590 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:33.173533 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.174165 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.174169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.174172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.174321 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.174750 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.174755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.174758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.175119 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:33.175268 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:33.175885 (+ 617us) server_negotiation.cc:299] Negotiation successful
0810 06:50:33.176104 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:50:33.177228 341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.160107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:41043 (local address 127.22.11.130:38643)
0810 06:50:33.160844 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:33.160888 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:33.162457 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:33.163031 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:33.163039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:33.163533 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:33.164088 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:33.164105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.165076 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.165079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:33.166271 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:33.166284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.167021 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.167029 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:33.167320 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.167992 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:33.168020 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:33.170330 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:33.173658 (+ 3328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:33.173665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:33.173675 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:33.174060 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:33.174410 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:33.174413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:33.174415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:33.174597 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:33.175226 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:33.175230 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:33.175529 (+ 299us) client_negotiation.cc:769] Sending connection context
0810 06:50:33.175738 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 06:50:33.176083 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":275,"threads_started":1}
I20260810 06:50:33.178664 335 heartbeater.cc:347] Connected to a master server at 127.22.11.190:41043
I20260810 06:50:33.178983 335 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:33.179699 335 heartbeater.cc:511] Master 127.22.11.190:41043 requested a full tablet report, sending...
I20260810 06:50:33.181089 32241 ts_manager.cc:194] Registered new tserver with Master: 72dc633a01b74bebba881fbdf4e61c5d (127.22.11.130:35049)
I20260810 06:50:33.181867 32241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:38643
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:50:33.216481 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:41043
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:33.531754 346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:33.532030 346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:33.532166 346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:33.539219 346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:33.539337 346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:33.539489 346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:33.549099 346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:41043
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:33.550346 346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:33.551636 346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:33.563863 355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:33.563910 356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:33.564052 358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:33.564851 346 server_base.cc:1034] running on GCE node
I20260810 06:50:33.565397 346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:33.566224 346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:33.567415 346 hybrid_clock.cc:648] HybridClock initialized: now 1786344633567368 us; error 40 us; skew 500 ppm
I20260810 06:50:33.567700 346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.573345 346 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:50:33.574651 346 webserver.cc:460] Webserver started at http://127.22.11.131:44301/ using document root <none> and password file <none>
I20260810 06:50:33.575245 346 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:33.575353 346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:33.575610 346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:33.578357 346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "d630e8f043304ee0b289c2c0ef6f4a9e"
format_stamp: "Formatted at 2026-08-10 06:50:33 on dist-test-slave-f7th"
server_key: "fa87be907a79bf9c27cf7653ef20cef7"
server_key_iv: "58e79735f33974fca71c186b792bbc20"
server_key_version: "encryptionkey@0"
I20260810 06:50:33.579066 346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "d630e8f043304ee0b289c2c0ef6f4a9e"
format_stamp: "Formatted at 2026-08-10 06:50:33 on dist-test-slave-f7th"
server_key: "fa87be907a79bf9c27cf7653ef20cef7"
server_key_iv: "58e79735f33974fca71c186b792bbc20"
server_key_version: "encryptionkey@0"
I20260810 06:50:33.584767 346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 06:50:33.587893 370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:33.589233 346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:33.589403 346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "d630e8f043304ee0b289c2c0ef6f4a9e"
format_stamp: "Formatted at 2026-08-10 06:50:33 on dist-test-slave-f7th"
server_key: "fa87be907a79bf9c27cf7653ef20cef7"
server_key_iv: "58e79735f33974fca71c186b792bbc20"
server_key_version: "encryptionkey@0"
I20260810 06:50:33.589615 346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:33.604323 346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:33.608273 346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:33.608520 346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:33.609416 346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:33.610662 346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:33.610769 346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:33.610895 346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:33.610954 346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:33.640486 346 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:35005
I20260810 06:50:33.640528 553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:35005 every 8 connection(s)
I20260810 06:50:33.641767 346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:33.644872 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 346
I20260810 06:50:33.645045 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:33.645637 22574 external_mini_cluster.cc:1428] Setting key d0ad94ba505395b60de55c79c50ae4dd
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:33.661090 32569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.644160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:38959 (local address 127.22.11.190:41043)
0810 06:50:33.644373 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.644378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.645733 (+ 1355us) server_negotiation.cc:407] Connection header received
0810 06:50:33.648092 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.648098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.648184 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.648281 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.649844 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.650584 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.652051 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.652449 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.652537 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.655713 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:33.655718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:33.655721 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:33.655752 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:33.658068 (+ 2316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.658702 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.658705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.658707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.658825 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.659278 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.659282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.659284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.659592 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:33.659729 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:33.660594 (+ 865us) server_negotiation.cc:299] Negotiation successful
0810 06:50:33.660843 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:50:33.661902 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.644678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:41043 (local address 127.22.11.131:38959)
0810 06:50:33.645558 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:33.645611 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:33.647827 (+ 2216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:33.648535 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:33.648543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:33.649166 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:33.649706 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:33.649724 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.650724 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.650727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:33.651938 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:33.651950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.652671 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.652680 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:33.652911 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.653514 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:33.653537 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:33.655545 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:33.658190 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:33.658197 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:33.658208 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:33.658613 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:33.658920 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:33.658925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:33.658928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:33.659167 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:33.659743 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:33.659750 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:33.660153 (+ 403us) client_negotiation.cc:769] Sending connection context
0810 06:50:33.660437 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 06:50:33.660912 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":366,"threads_started":1}
I20260810 06:50:33.663046 557 heartbeater.cc:347] Connected to a master server at 127.22.11.190:41043
I20260810 06:50:33.663318 557 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:33.663980 557 heartbeater.cc:511] Master 127.22.11.190:41043 requested a full tablet report, sending...
I20260810 06:50:33.665284 32241 ts_manager.cc:194] Registered new tserver with Master: d630e8f043304ee0b289c2c0ef6f4a9e (127.22.11.131:35005)
I20260810 06:50:33.666049 32241 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:38959
I20260810 06:50:33.672680 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/test-admin.keytab.
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.703351 22574 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
I20260810 06:50:33.705416 32565 heartbeater.cc:503] Master 127.22.11.190:41043 was elected leader, sending a full tablet report...
W20260810 06:50:33.708536 571 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:33.705979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60478 (local address 127.22.11.129:35085)
0810 06:50:33.706702 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.706708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.706732 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:50:33.706786 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.706792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.707003 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.707169 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.708165 (+ 996us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60478: BlockingRecv error: recv got EOF from 127.0.0.1:60478 (error 108)
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":388,"threads_started":1}
W20260810 06:50:33.708868 22574 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.710320 22574 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.711699 22574 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:33.725708 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.713217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.22.11.129:35085)
0810 06:50:33.713347 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.713351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.713701 (+ 350us) server_negotiation.cc:407] Connection header received
0810 06:50:33.713846 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.713850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.713902 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.713977 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.714912 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.715839 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.716991 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.717525 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.717619 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.720829 (+ 3210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:33.720840 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:33.720852 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:33.720890 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:33.723391 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.723959 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.723968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.723970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.724095 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.724365 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.724368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.724369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.724752 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:33.724976 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:33.725229 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 06:50:33.725427 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260810 06:50:33.730880 571 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:33.729313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.22.11.129:35085)
0810 06:50:33.729484 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.729490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.729732 (+ 242us) server_negotiation.cc:407] Connection header received
0810 06:50:33.729872 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.729877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.729927 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.729999 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.730683 (+ 684us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60490: BlockingRecv error: recv got EOF from 127.0.0.1:60490 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 06:50:33.731047 22574 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.732355 22574 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.733616 22574 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:33.745402 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.734673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60498 (local address 127.22.11.129:35085)
0810 06:50:33.734848 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.734853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.735080 (+ 227us) server_negotiation.cc:407] Connection header received
0810 06:50:33.735179 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.735183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.735230 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.735299 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.736082 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.736776 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.737864 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.738253 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.738335 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.741268 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:33.741273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:33.741276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:33.741316 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:33.743451 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.743973 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.743976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.743978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.744082 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.744340 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.744343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.744344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.744575 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:33.744822 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:33.744994 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 06:50:33.745140 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 06:50:33.748780 571 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:33.747478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.22.11.129:35085)
0810 06:50:33.747652 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.747656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.747824 (+ 168us) server_negotiation.cc:407] Connection header received
0810 06:50:33.747919 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.747922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.747980 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.748055 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.748574 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60502: BlockingRecv error: recv got EOF from 127.0.0.1:60502 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 06:50:33.748983 22574 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786344580063516-22574-0/krb5kdc/krb5cc' of type FILE
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.750243 22574 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:33.751360 22574 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:33.763492 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.752391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.22.11.129:35085)
0810 06:50:33.752577 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.752582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.752806 (+ 224us) server_negotiation.cc:407] Connection header received
0810 06:50:33.752898 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.752901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.752946 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.753013 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.753836 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.754472 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.755544 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.755938 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.756019 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.759130 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:33.759135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:33.759138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:33.759165 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:33.761519 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.762041 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.762044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.762047 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.762190 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.762451 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.762454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.762456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.762697 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:33.762909 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:33.763039 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 06:50:33.763193 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:50:33.764835 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 32339
I20260810 06:50:33.775481 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.765526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60526 (local address 127.22.11.129:35085)
0810 06:50:33.765686 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.765692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.765915 (+ 223us) server_negotiation.cc:407] Connection header received
0810 06:50:33.766051 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.766056 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.766111 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.766183 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.767017 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.767652 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.768825 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.769212 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.769322 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.771170 (+ 1848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:33.771174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:33.771177 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:33.771205 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:33.773260 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.773795 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.773800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.773804 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.773921 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.774262 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.774267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.774270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.774503 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:33.774730 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:33.775028 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 06:50:33.775210 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:50:33.776038 32558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:33.858029 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 32339
I20260810 06:50:33.880420 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 32576
Aug 10 06:50:33 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:50:33.895565 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:33.881305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.22.11.130:35049)
0810 06:50:33.881830 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:33.881838 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:33.881862 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:50:33.881923 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:33.881929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:33.882143 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:33.882292 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:33.883299 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.884311 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.885648 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:33.886196 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:33.886311 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:33.890310 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:33.890322 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:33.890336 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:33.890368 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:33.892987 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.893680 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.893687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.893689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.893809 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:33.894105 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:33.894107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:33.894109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:33.894481 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:33.894757 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:33.895002 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 06:50:33.895233 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":280,"threads_started":1}
I20260810 06:50:33.896876 322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:33.977361 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 32576
I20260810 06:50:34.001305 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 346
Aug 10 06:50:34 dist-test-slave-f7th krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:50:34.016192 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:34.002278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39318 (local address 127.22.11.131:35005)
0810 06:50:34.002890 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:34.002897 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:34.002928 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:50:34.002998 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:34.003004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:34.003214 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:34.003335 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:34.004378 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:34.005526 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:34.006725 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:34.007288 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:34.007391 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:34.011218 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:34.011229 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:34.011245 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:34.011278 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:34.013846 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:34.014474 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:34.014481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:34.014483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:34.014604 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:34.014891 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:34.014894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:34.014896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:34.015232 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:34.015476 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:34.015644 (+ 168us) server_negotiation.cc:299] Negotiation successful
0810 06:50:34.015871 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":289,"threads_started":1}
I20260810 06:50:34.017473 546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:34.098712 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 346
I20260810 06:50:34.121505 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 32176
I20260810 06:50:34.122637 32292 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:34.223543 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 32176
2026-08-10T06:50:34Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4017 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:34 dist-test-slave-f7th krb5kdc[603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:34 dist-test-slave-f7th krb5kdc[603](info): set up 2 sockets
Aug 10 06:50:34 dist-test-slave-f7th krb5kdc[603](info): commencing operation
krb5kdc: starting...
W20260810 06:50:35.477387 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:35 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:35Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab.
Entry for principal oryx/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab.
I20260810 06:50:35.692122 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:38069
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab
--principal=oryx/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:38069
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:36.010294 620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:36.010555 620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:36.010684 620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:36.017661 620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:36.017757 620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:36.017850 620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:36.017938 620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:36.017992 620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:36.027314 620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:38069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab
--principal=oryx/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:38069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:36.028544 620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:36.029824 620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:36.040583 632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:36.040700 629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:36.040664 630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:36.041531 620 server_base.cc:1034] running on GCE node
I20260810 06:50:36.042234 620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:36.043488 620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:36.044708 620 hybrid_clock.cc:648] HybridClock initialized: now 1786344636044653 us; error 36 us; skew 500 ppm
I20260810 06:50:36.045025 620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:36 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344636, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:36.047974 620 init.cc:377] Logged in from keytab as oryx/127.22.11.190@KRBTEST.COM (short username oryx)
I20260810 06:50:36.049328 620 webserver.cc:460] Webserver started at http://127.22.11.190:41343/ using document root <none> and password file <none>
I20260810 06:50:36.049965 620 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:36.050084 620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:36.050369 620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:36.053210 620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "f1d212eb435b4fe99b8df6f621429257"
format_stamp: "Formatted at 2026-08-10 06:50:36 on dist-test-slave-f7th"
server_key: "8713f08937ecfd9fdd2f6344721dfbd0"
server_key_iv: "907f6396c42c229e54d25d52b171546b"
server_key_version: "encryptionkey@0"
I20260810 06:50:36.053946 620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "f1d212eb435b4fe99b8df6f621429257"
format_stamp: "Formatted at 2026-08-10 06:50:36 on dist-test-slave-f7th"
server_key: "8713f08937ecfd9fdd2f6344721dfbd0"
server_key_iv: "907f6396c42c229e54d25d52b171546b"
server_key_version: "encryptionkey@0"
I20260810 06:50:36.058755 620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:50:36.061961 642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:36.063205 620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:36.063385 620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "f1d212eb435b4fe99b8df6f621429257"
format_stamp: "Formatted at 2026-08-10 06:50:36 on dist-test-slave-f7th"
server_key: "8713f08937ecfd9fdd2f6344721dfbd0"
server_key_iv: "907f6396c42c229e54d25d52b171546b"
server_key_version: "encryptionkey@0"
I20260810 06:50:36.063562 620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:36.081290 620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:36.084785 620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:36.085026 620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:36.101408 620 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:38069
I20260810 06:50:36.101440 740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:38069 every 8 connection(s)
I20260810 06:50:36.102679 620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:36.106175 743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:36.110435 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 620
I20260810 06:50:36.110653 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:36.111509 22574 external_mini_cluster.cc:1428] Setting key ad39daa31dc6d7b5f705496e5837d1fa
I20260810 06:50:36.117547 743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: Bootstrap starting.
I20260810 06:50:36.120987 743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:36.122102 743 log.cc:824] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:36.125173 743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: No bootstrap required, opened a new log
Aug 10 06:50:36 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:36.132926 743 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } }
I20260810 06:50:36.133270 743 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:36.133371 743 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d212eb435b4fe99b8df6f621429257, State: Initialized, Role: FOLLOWER
I20260810 06:50:36.133785 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:36.113493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32988 (local address 127.22.11.190:38069)
0810 06:50:36.114261 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:36.114271 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:36.114311 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:50:36.116049 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:36.116070 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:36.116403 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:36.116752 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:36.118496 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:36.119783 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:36.121473 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:36.122115 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:36.122272 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:36.126718 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:36.126725 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:36.126741 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:36.126778 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:36.129906 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:36.130551 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:36.130558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:36.130564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:36.130731 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:36.131104 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:36.131107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:36.131109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:36.131640 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:36.131956 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:36.132371 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 06:50:36.132776 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":334,"threads_started":1}
I20260810 06:50:36.133985 743 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } }
I20260810 06:50:36.134279 743 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:36.134424 743 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:36.134608 743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:36.137118 743 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } }
I20260810 06:50:36.137539 743 leader_election.cc:302] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1d212eb435b4fe99b8df6f621429257; no voters:
I20260810 06:50:36.138018 743 leader_election.cc:288] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:36.138135 756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:36.138774 756 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 1 LEADER]: Becoming Leader. State: Replica: f1d212eb435b4fe99b8df6f621429257, State: Running, Role: LEADER
I20260810 06:50:36.139277 756 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } }
I20260810 06:50:36.139811 743 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:36.142205 758 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d212eb435b4fe99b8df6f621429257. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } }
I20260810 06:50:36.142266 757 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } }
I20260810 06:50:36.142581 758 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:36.142622 757 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:36.144330 767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:36.149475 767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:36.157765 767 catalog_manager.cc:1347] Generated new cluster ID: 4d8b3452f15a4a3ea68023252ba6e2d8
I20260810 06:50:36.157867 767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:36.166297 767 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:36.167727 767 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:36.175251 767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: Generated new TSK 0
I20260810 06:50:36.176044 767 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab.
Entry for principal oryx/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab.
I20260810 06:50:36.233927 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:38069
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab
--principal=oryx/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:36.555377 787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:36.555624 787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:36.555753 787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:36.562713 787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:36.562808 787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:36.562960 787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:36.572336 787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab
--principal=oryx/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:36.573505 787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:36.574759 787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:36.586637 797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:36.586714 800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:36.586665 795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:36.587823 787 server_base.cc:1034] running on GCE node
I20260810 06:50:36.588423 787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:36.589311 787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:36.590517 787 hybrid_clock.cc:648] HybridClock initialized: now 1786344636590459 us; error 43 us; skew 500 ppm
I20260810 06:50:36.590869 787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:36 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344636, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:36.594323 787 init.cc:377] Logged in from keytab as oryx/127.22.11.129@KRBTEST.COM (short username oryx)
I20260810 06:50:36.595692 787 webserver.cc:460] Webserver started at http://127.22.11.129:34277/ using document root <none> and password file <none>
I20260810 06:50:36.596334 787 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:36.596452 787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:36.596776 787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:36.599602 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "7da9d94e1ade47dabb9975696e23b7c1"
format_stamp: "Formatted at 2026-08-10 06:50:36 on dist-test-slave-f7th"
server_key: "d19b75e58d5e341892a51ddcb00ccd06"
server_key_iv: "8b80fd5e38036b2a3d706fa739fd143c"
server_key_version: "encryptionkey@0"
I20260810 06:50:36.600353 787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "7da9d94e1ade47dabb9975696e23b7c1"
format_stamp: "Formatted at 2026-08-10 06:50:36 on dist-test-slave-f7th"
server_key: "d19b75e58d5e341892a51ddcb00ccd06"
server_key_iv: "8b80fd5e38036b2a3d706fa739fd143c"
server_key_version: "encryptionkey@0"
I20260810 06:50:36.605336 787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:50:36.608598 813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:36.609812 787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:50:36.609965 787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "7da9d94e1ade47dabb9975696e23b7c1"
format_stamp: "Formatted at 2026-08-10 06:50:36 on dist-test-slave-f7th"
server_key: "d19b75e58d5e341892a51ddcb00ccd06"
server_key_iv: "8b80fd5e38036b2a3d706fa739fd143c"
server_key_version: "encryptionkey@0"
I20260810 06:50:36.610143 787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:36.637853 787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:36.641734 787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:36.641991 787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:36.642881 787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:36.644214 787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:36.644342 787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:36.644444 787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:36.644488 787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:36.674649 787 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:33747
I20260810 06:50:36.674686 1019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:33747 every 8 connection(s)
I20260810 06:50:36.675890 787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:36.685225 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 787
I20260810 06:50:36.685431 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:36.686223 22574 external_mini_cluster.cc:1428] Setting key fbb15fcfa7741e32b88f37f69a26e72c
Aug 10 06:50:36 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344636, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:36.694713 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:36.678285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:48817 (local address 127.22.11.190:38069)
0810 06:50:36.678713 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:36.678720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:36.679658 (+ 938us) server_negotiation.cc:407] Connection header received
0810 06:50:36.681430 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:36.681435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:36.681502 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:36.681587 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:36.682927 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:36.683640 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:36.685184 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:36.685769 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:36.685896 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:36.689394 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:36.689399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:36.689402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:36.689433 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:36.691807 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:36.692593 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:36.692597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:36.692599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:36.692749 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:36.693177 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:36.693180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:36.693182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:36.693420 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:36.693526 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:36.694280 (+ 754us) server_negotiation.cc:299] Negotiation successful
0810 06:50:36.694442 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":196,"threads_started":1}
I20260810 06:50:36.695932 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:36.678816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:48817)
0810 06:50:36.679504 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:36.679552 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:36.681187 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:36.681766 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:36.681774 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:36.682252 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:36.682798 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:36.682814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:36.683757 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:36.683760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:36.685035 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:36.685050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:36.685963 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:36.685968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:36.686157 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:36.686781 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:36.686810 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:36.689193 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:36.691939 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:36.691947 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:36.691961 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:36.692482 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:36.692846 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:36.692850 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:36.692853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:36.693085 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:36.693526 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:36.693533 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:36.693906 (+ 373us) client_negotiation.cc:769] Sending connection context
0810 06:50:36.694201 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 06:50:36.694679 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":253,"threads_started":1}
I20260810 06:50:36.697346 1020 heartbeater.cc:347] Connected to a master server at 127.22.11.190:38069
I20260810 06:50:36.697634 1020 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:36.698351 1020 heartbeater.cc:511] Master 127.22.11.190:38069 requested a full tablet report, sending...
I20260810 06:50:36.700351 674 ts_manager.cc:194] Registered new tserver with Master: 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:36.701831 674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:48817
WARNING: no policy specified for oryx/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab.
Entry for principal oryx/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab.
I20260810 06:50:36.736426 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:38069
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab
--principal=oryx/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:37.061152 1029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:37.061434 1029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:37.061576 1029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:37.068711 1029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:37.068828 1029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:37.068985 1029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:37.078227 1029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab
--principal=oryx/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:37.079427 1029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:37.080730 1029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:37.093114 1045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:37.093158 1038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:37.093190 1042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:37.094135 1029 server_base.cc:1034] running on GCE node
I20260810 06:50:37.094751 1029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:37.095641 1029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:37.096840 1029 hybrid_clock.cc:648] HybridClock initialized: now 1786344637096770 us; error 65 us; skew 500 ppm
I20260810 06:50:37.097146 1029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:37 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344637, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:37.100186 1029 init.cc:377] Logged in from keytab as oryx/127.22.11.130@KRBTEST.COM (short username oryx)
I20260810 06:50:37.101846 1029 webserver.cc:460] Webserver started at http://127.22.11.130:45699/ using document root <none> and password file <none>
I20260810 06:50:37.102667 1029 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:37.102802 1029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:37.103152 1029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:37.106927 1029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "b74fa238fb9948a580de569f784c290f"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "db7c902160f4e8c50b2acefc24563f6e"
server_key_iv: "d7ce59a54c828f160a285f528b296fab"
server_key_version: "encryptionkey@0"
I20260810 06:50:37.107923 1029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "b74fa238fb9948a580de569f784c290f"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "db7c902160f4e8c50b2acefc24563f6e"
server_key_iv: "d7ce59a54c828f160a285f528b296fab"
server_key_version: "encryptionkey@0"
I20260810 06:50:37.114315 1029 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 06:50:37.118633 1055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:37.120141 1029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:37.120306 1029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "b74fa238fb9948a580de569f784c290f"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "db7c902160f4e8c50b2acefc24563f6e"
server_key_iv: "d7ce59a54c828f160a285f528b296fab"
server_key_version: "encryptionkey@0"
I20260810 06:50:37.120518 1029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:37.133426 1029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:37.137315 1029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:37.137548 1029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:37.138377 1029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:37.139669 1029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:37.139739 1029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:37.139832 1029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:37.139874 1029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:37.169488 1029 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:33987
I20260810 06:50:37.169524 1232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:33987 every 8 connection(s)
I20260810 06:50:37.170756 1029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:37.175822 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 1029
I20260810 06:50:37.175976 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:37.176427 22574 external_mini_cluster.cc:1428] Setting key f156ba0b4adec2ef2100e4d60e7c1544
Aug 10 06:50:37 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344637, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:37.190660 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:37.173156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:37463 (local address 127.22.11.190:38069)
0810 06:50:37.173321 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:37.173325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:37.174396 (+ 1071us) server_negotiation.cc:407] Connection header received
0810 06:50:37.176565 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:37.176571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:37.176686 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:37.176776 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:37.178513 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.179207 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.180731 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.181132 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.181247 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:37.184590 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:37.184597 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:37.184601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:37.184665 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:37.187601 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:37.188427 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:37.188430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:37.188432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:37.188549 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:37.189025 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:37.189028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:37.189030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:37.189281 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:37.189386 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:37.190165 (+ 779us) server_negotiation.cc:299] Negotiation successful
0810 06:50:37.190388 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:50:37.191807 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:37.173573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:37463)
0810 06:50:37.174243 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:37.174287 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:37.176293 (+ 2006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:37.177044 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:37.177052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:37.177676 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:37.178354 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:37.178374 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.179360 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.179364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:37.180573 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:37.180587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.181350 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.181358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:37.181580 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:37.182278 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:37.182307 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:37.184366 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:37.187761 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:37.187770 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:37.187785 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:37.188319 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:37.188667 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:37.188672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:37.188675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:37.188928 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:37.189411 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:37.189419 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:37.189782 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 06:50:37.190067 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 06:50:37.190544 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":247,"threads_started":1}
I20260810 06:50:37.193114 1236 heartbeater.cc:347] Connected to a master server at 127.22.11.190:38069
I20260810 06:50:37.193400 1236 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:37.194095 1236 heartbeater.cc:511] Master 127.22.11.190:38069 requested a full tablet report, sending...
I20260810 06:50:37.195485 674 ts_manager.cc:194] Registered new tserver with Master: b74fa238fb9948a580de569f784c290f (127.22.11.130:33987)
I20260810 06:50:37.196249 674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:37463
WARNING: no policy specified for oryx/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab.
Entry for principal oryx/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab.
I20260810 06:50:37.227514 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:38069
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab
--principal=oryx/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:37.549389 1250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:37.549649 1250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:37.549783 1250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:37.556756 1250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:37.556854 1250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:37.557019 1250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:37.566215 1250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab
--principal=oryx/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:38069
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:37.567515 1250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:37.568836 1250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:37.580679 1260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:37.580678 1261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:37.580721 1263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:37.581797 1250 server_base.cc:1034] running on GCE node
I20260810 06:50:37.582340 1250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:37.583372 1250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:37.584565 1250 hybrid_clock.cc:648] HybridClock initialized: now 1786344637584510 us; error 50 us; skew 500 ppm
I20260810 06:50:37.584867 1250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:37 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344637, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:37.587818 1250 init.cc:377] Logged in from keytab as oryx/127.22.11.131@KRBTEST.COM (short username oryx)
I20260810 06:50:37.589165 1250 webserver.cc:460] Webserver started at http://127.22.11.131:40425/ using document root <none> and password file <none>
I20260810 06:50:37.589807 1250 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:37.589923 1250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:37.590210 1250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:37.593329 1250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "36b16974d96a42b7bf934f165eef02c2"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "21561ce9f76abea2f81e3fd44d80a27b"
server_key_iv: "3cae663d334e5ffe6ee5ac243460a2d3"
server_key_version: "encryptionkey@0"
I20260810 06:50:37.594381 1250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "36b16974d96a42b7bf934f165eef02c2"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "21561ce9f76abea2f81e3fd44d80a27b"
server_key_iv: "3cae663d334e5ffe6ee5ac243460a2d3"
server_key_version: "encryptionkey@0"
I20260810 06:50:37.599639 1250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:50:37.602865 1281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:37.604108 1250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:50:37.604257 1250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "36b16974d96a42b7bf934f165eef02c2"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "21561ce9f76abea2f81e3fd44d80a27b"
server_key_iv: "3cae663d334e5ffe6ee5ac243460a2d3"
server_key_version: "encryptionkey@0"
I20260810 06:50:37.604434 1250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:37.617084 1250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:37.620572 1250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:37.620836 1250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:37.621673 1250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:37.622882 1250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:37.622959 1250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:37.623075 1250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:37.623129 1250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:37.653026 1250 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:43297
I20260810 06:50:37.653069 1479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:43297 every 8 connection(s)
I20260810 06:50:37.654310 1250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:37.658811 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 1250
I20260810 06:50:37.658990 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:37.659545 22574 external_mini_cluster.cc:1428] Setting key 0b7c36c3dd409488d23415fe67aa8851
Aug 10 06:50:37 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344637, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:37.673586 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:37.656775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:45585 (local address 127.22.11.190:38069)
0810 06:50:37.656951 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:37.656956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:37.658109 (+ 1153us) server_negotiation.cc:407] Connection header received
0810 06:50:37.660331 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:37.660336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:37.660438 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:37.660541 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:37.662304 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.663103 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.664642 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.665051 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.665146 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:37.668385 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:37.668391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:37.668396 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:37.668428 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:37.670827 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:37.671435 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:37.671439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:37.671441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:37.671562 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:37.671926 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:37.671929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:37.671931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:37.672216 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:37.672360 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:37.673080 (+ 720us) server_negotiation.cc:299] Negotiation successful
0810 06:50:37.673248 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:50:37.674716 1482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:37.657241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:45585)
0810 06:50:37.657916 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:37.657956 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:37.660071 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:37.660789 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:37.660800 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:37.661514 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:37.662132 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:37.662151 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.663232 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.663236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:37.664477 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:37.664489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:37.665220 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:37.665225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:37.665411 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:37.666092 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:37.666114 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:37.668200 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:37.670940 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:37.670946 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:37.670957 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:37.671352 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:37.671638 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:37.671641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:37.671643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:37.671809 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:37.672325 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:37.672331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:37.672607 (+ 276us) client_negotiation.cc:769] Sending connection context
0810 06:50:37.672939 (+ 332us) client_negotiation.cc:240] Negotiation successful
0810 06:50:37.673392 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":264,"threads_started":1}
I20260810 06:50:37.676051 1480 heartbeater.cc:347] Connected to a master server at 127.22.11.190:38069
I20260810 06:50:37.676354 1480 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:37.677104 1480 heartbeater.cc:511] Master 127.22.11.190:38069 requested a full tablet report, sending...
I20260810 06:50:37.678432 674 ts_manager.cc:194] Registered new tserver with Master: 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297)
I20260810 06:50:37.679190 674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:45585
I20260810 06:50:37.686583 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:50:37.705025 1020 heartbeater.cc:503] Master 127.22.11.190:38069 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.22.11.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.189.keytab.
Entry for principal oryx/127.22.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.189.keytab.
I20260810 06:50:37.737773 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:41675
--webserver_interface=127.22.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.189.keytab
--principal=oryx/127.22.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:38069,127.22.11.189:41675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:38.061298 1489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:38.061556 1489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:38.061654 1489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:38.068588 1489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:38.068710 1489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:38.068768 1489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:38.068823 1489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:38.068872 1489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:38.078195 1489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:38069,127.22.11.189:41675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.189.keytab
--principal=oryx/127.22.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:41675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
W20260810 06:50:38.078473 1489 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.22.11.190:38069,127.22.11.189:41675'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 06:50:38.079420 1489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:38.080662 1489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:38.091955 1498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:38.091993 1501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:38.092032 1499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:38.092909 1489 server_base.cc:1034] running on GCE node
I20260810 06:50:38.093633 1489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:38.094918 1489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:38.096144 1489 hybrid_clock.cc:648] HybridClock initialized: now 1786344638096069 us; error 44 us; skew 500 ppm
I20260810 06:50:38.096446 1489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:38.099421 1489 init.cc:377] Logged in from keytab as oryx/127.22.11.189@KRBTEST.COM (short username oryx)
I20260810 06:50:38.100813 1489 webserver.cc:460] Webserver started at http://127.22.11.189:44163/ using document root <none> and password file <none>
I20260810 06:50:38.101634 1489 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:38.101758 1489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:38.102133 1489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:38.105886 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/instance:
uuid: "bb98122a9ca4498bb89cd5c70007e702"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "db978a8630540c515538985abfad9aaf"
server_key_iv: "5a1b3d862549c0f50f05326717887b4d"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.106804 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal/instance:
uuid: "bb98122a9ca4498bb89cd5c70007e702"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "db978a8630540c515538985abfad9aaf"
server_key_iv: "5a1b3d862549c0f50f05326717887b4d"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.112890 1489 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 06:50:38.116155 1511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:38.117451 1489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:38.117612 1489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
uuid: "bb98122a9ca4498bb89cd5c70007e702"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "db978a8630540c515538985abfad9aaf"
server_key_iv: "5a1b3d862549c0f50f05326717887b4d"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.117805 1489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:38.141762 1489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:38.145504 1489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:38.145741 1489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:38.162819 1489 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.189:41675
I20260810 06:50:38.162838 1588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.189:41675 every 8 connection(s)
I20260810 06:50:38.164065 1489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
I20260810 06:50:38.166358 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 1489
I20260810 06:50:38.167970 1589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:38.177585 1589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.189@KRBTEST.COM
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.189@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:38.190249 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.168420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.22.11.189:41675)
0810 06:50:38.169358 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.169372 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.169419 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:50:38.171660 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.171685 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.172172 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.172742 (+ 570us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.174421 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.175857 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.177518 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.178257 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.178416 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.182910 (+ 4494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.182917 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.182926 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.182962 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.186305 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.187066 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.187075 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.187083 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.187293 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.187636 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.187645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.187648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.188215 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.188435 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.188937 (+ 502us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.189291 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":382,"threads_started":1}
I20260810 06:50:38.195152 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.180213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.22.11.190:38069)
0810 06:50:38.181522 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.181527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.181549 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:50:38.181597 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.181600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.181655 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.181735 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.182884 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.183695 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.185394 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.185784 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.185879 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.189373 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.189378 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.189381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.189412 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.192970 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.193592 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.193596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.193598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.193715 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.194026 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.194031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.194033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.194313 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.194419 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.194755 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.194910 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1199,"thread_start_us":1074,"threads_started":1}
I20260810 06:50:38.195163 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.180160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32992)
0810 06:50:38.180929 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.180970 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.181114 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.181835 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.181841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.182494 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.182724 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.182737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.183837 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.183846 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.185246 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.185262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.185971 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.185982 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.186169 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.186765 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.186782 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.189199 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.193088 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.193097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.193101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.193504 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.193788 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.193791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.193793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.193935 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.194403 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.194408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.194543 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.194678 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.194910 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":365,"threads_started":1}
I20260810 06:50:38.196796 1589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:50:38.198580 1236 heartbeater.cc:503] Master 127.22.11.190:38069 was elected leader, sending a full tablet report...
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.189@KRBTEST.COM for oryx/127.22.11.189@KRBTEST.COM
I20260810 06:50:38.207787 1594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.197318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.0.0.1:52116)
0810 06:50:38.197493 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.197514 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.197599 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.197869 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.197872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.198161 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.198279 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.198287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.199279 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.199283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.200314 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.200326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.201007 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.201012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.201147 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.201605 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.201621 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.203388 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.205896 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.205900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.205902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.206242 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.206558 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.206561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.206563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.206675 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.207113 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.207116 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.207232 (+ 116us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.207332 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.207542 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 06:50:38.208009 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.197555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.22.11.189:41675)
0810 06:50:38.197673 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.197676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.197689 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:50:38.197723 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.197726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.197772 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.197841 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.198396 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.199154 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.200430 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.200885 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.200972 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.203539 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.203543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.203546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.203573 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.205790 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.206330 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.206333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.206335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.206463 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.206757 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.206760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.206762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.207020 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.207121 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.207412 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.207662 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":38}
I20260810 06:50:38.212198 1589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Bootstrap starting.
I20260810 06:50:38.215332 1589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:38.216316 1589 log.cc:824] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:38.218780 1589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: No bootstrap required, opened a new log
I20260810 06:50:38.225859 1589 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } }
I20260810 06:50:38.226142 1589 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:38.226233 1589 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb98122a9ca4498bb89cd5c70007e702, State: Initialized, Role: FOLLOWER
I20260810 06:50:38.226943 1589 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } }
I20260810 06:50:38.227260 1599 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:38.227557 1599 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } }
I20260810 06:50:38.228258 1601 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } } }
I20260810 06:50:38.228734 1599 leader_election.cc:288] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069)
I20260810 06:50:38.229003 1601 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:50:38.229074 1589 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:38.229692 695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb98122a9ca4498bb89cd5c70007e702" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1d212eb435b4fe99b8df6f621429257" is_pre_election: true
I20260810 06:50:38.231104 1512 leader_election.cc:302] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: bb98122a9ca4498bb89cd5c70007e702; no voters: f1d212eb435b4fe99b8df6f621429257
I20260810 06:50:38.231598 1601 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 06:50:38.237550 1624 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:50:38.237653 1624 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:50:38.240552 1489 master_runner.cc:428] Detected that this master bb98122a9ca4498bb89cd5c70007e702 is joining an existing cluster
I20260810 06:50:38.240651 1489 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.11.189:41675
I20260810 06:50:38.268224 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.244783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.22.11.190:38069)
0810 06:50:38.244971 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.244978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.251529 (+ 6551us) server_negotiation.cc:407] Connection header received
0810 06:50:38.252068 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.252074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.252189 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.252275 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.253853 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.254939 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.256961 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.257570 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.257664 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.261417 (+ 3753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.261424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.261429 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.261467 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.264936 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.265844 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.265850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.265854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.266018 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.266510 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.266515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.266518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.266906 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.267038 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.267646 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.267887 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:50:38.269311 1638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.244484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33000)
0810 06:50:38.251428 (+ 6944us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.251448 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.251900 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.252498 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.252503 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.253238 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.253680 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.253693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.255121 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.255126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.256778 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.256795 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.257967 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.257974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.258163 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.260000 (+ 1837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.260026 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.261220 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.265096 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.265105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.265108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.265666 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.266199 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.266205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.266208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.266380 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.267117 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.267121 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.267325 (+ 204us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.268474 (+ 1149us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.268804 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6794,"thread_start_us":12289,"threads_started":1}
I20260810 06:50:38.271117 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.245113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.22.11.189:41675)
0810 06:50:38.245300 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.245305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.257134 (+ 11829us) server_negotiation.cc:407] Connection header received
0810 06:50:38.257263 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.257268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.257336 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.257428 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.258738 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.259770 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.261322 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.261938 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.262037 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.264212 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.264218 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.264221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.264253 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.267692 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.268833 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.268841 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.268844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.269015 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.269381 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.269385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.269387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.269733 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.269887 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.270615 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.270791 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260810 06:50:38.272190 1637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.244645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.0.0.1:52132)
0810 06:50:38.257026 (+ 12381us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.257052 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.257162 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.257571 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.257575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.258277 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.258605 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.258618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.259928 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.259932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.261187 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.261201 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.262318 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.262325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.262495 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.263027 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.263046 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.264056 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.267885 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.267889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.267892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.268722 (+ 830us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.269131 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.269134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.269137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.269289 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.270256 (+ 967us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.270260 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.270393 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.271514 (+ 1121us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.271764 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12246,"thread_start_us":6906,"threads_started":1}
WARNING: no policy specified for oryx/127.22.11.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.188@KRBTEST.COM" created.
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344636, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.190@KRBTEST.COM for oryx/127.22.11.189@KRBTEST.COM
I20260810 06:50:38.296423 1593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.276874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.22.11.189:41675)
0810 06:50:38.277050 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.277056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.279992 (+ 2936us) server_negotiation.cc:407] Connection header received
0810 06:50:38.280266 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.280272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.280382 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.280484 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.281592 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.282693 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.284416 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.285141 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.285269 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.289988 (+ 4719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.289995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.290000 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.290039 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.293383 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.294221 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.294227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.294230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.294407 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.294872 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.294876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.294879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.295229 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.295358 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.295902 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.296102 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:50:38.296507 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.276664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.0.0.1:52144)
0810 06:50:38.279856 (+ 3192us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.279900 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.280073 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.280516 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.280521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.281157 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.281425 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.281436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.282851 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.282857 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.284237 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.284252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.285305 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.285311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.285489 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.286736 (+ 1247us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.286755 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.289445 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.293517 (+ 4072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.293529 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.293544 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.294082 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.294546 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.294551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.294554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.294752 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.295357 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.295365 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.295566 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.295766 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.296079 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2991,"thread_start_us":2842,"threads_started":1}
I20260810 06:50:38.298839 674 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.11.189:41675: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702"
member_type: NON_VOTER
last_known_addr {
host: "127.22.11.189"
port: 41675
}
attrs {
promote: true
}
}
dest_uuid: "f1d212eb435b4fe99b8df6f621429257"
cas_config_opid_index: -1
I20260810 06:50:38.300529 674 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } }
I20260810 06:50:38.305371 1647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bb98122a9ca4498bb89cd5c70007e702 (127.22.11.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } }
I20260810 06:50:38.306739 1558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:38.309835 1647 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.11.189:41675
I20260810 06:50:38.310310 1558 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Refusing update from remote peer f1d212eb435b4fe99b8df6f621429257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 06:50:38.310379 1647 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } }
I20260810 06:50:38.310616 1647 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:38.310801 1489 master.cc:537] Master@127.22.11.189:41675 shutting down...
I20260810 06:50:38.311415 1651 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 06:50:38.314894 649 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.188.keytab.
Entry for principal oryx/127.22.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.188.keytab.
I20260810 06:50:38.317684 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:42527
--webserver_interface=127.22.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.188.keytab
--principal=oryx/127.22.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:50:38.333634 1489 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:38.334103 1489 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:38.334359 1489 tablet_replica.cc:330] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: stopping tablet replica
W20260810 06:50:38.335192 649 consensus_peers.cc:487] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 -> Peer bb98122a9ca4498bb89cd5c70007e702 (127.22.11.189:41675): Couldn't send request to peer bb98122a9ca4498bb89cd5c70007e702. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:50:38.350890 1489 master.cc:552] Master@127.22.11.189:41675 shutdown complete.
I20260810 06:50:38.350976 1489 master_runner.cc:305] Clearing existing system tablet
I20260810 06:50:38.351438 1489 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:38.351543 1489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:38.354527 1489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260810 06:50:38.356140 1657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:38.356494 1489 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:38.356802 1489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260810 06:50:38.356895 1489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
uuid: "bb98122a9ca4498bb89cd5c70007e702"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "db978a8630540c515538985abfad9aaf"
server_key_iv: "5a1b3d862549c0f50f05326717887b4d"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.362056 1489 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:38.368899 1489 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:50:38.369071 1489 log.cc:1196] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 06:50:38.369503 1489 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Deleting consensus metadata
I20260810 06:50:38.369781 1489 master_runner.cc:315] Copying system tablet from 127.22.11.190:38069
I20260810 06:50:38.372562 1489 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: tablet copy: Beginning tablet copy session from remote peer at address 127.22.11.190:38069
I20260810 06:50:38.391363 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.373550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.22.11.190:38069)
0810 06:50:38.373728 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.373735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.377359 (+ 3624us) server_negotiation.cc:407] Connection header received
0810 06:50:38.377544 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.377550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.377660 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.377778 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.378836 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.379957 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.381813 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.382459 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.382609 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.384919 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.384926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.384931 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.384971 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.388390 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.389190 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.389195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.389199 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.389361 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.389764 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.389768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.389771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.390156 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.390313 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.390783 (+ 470us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.391004 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:50:38.391383 1669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.373529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33002)
0810 06:50:38.377212 (+ 3683us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.377239 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.377407 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.377816 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.377821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.378364 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.378646 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.378664 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.380118 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.380123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.381630 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.381646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.382619 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.382625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.382810 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.383495 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.383520 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.384638 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.388523 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.388529 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.388533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.389058 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.389475 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.389480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.389484 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.389648 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.390280 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.390285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.390477 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.390642 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.390937 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3536,"thread_start_us":3413,"threads_started":1}
I20260810 06:50:38.392706 713 tablet_copy_service.cc:140] P f1d212eb435b4fe99b8df6f621429257: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bb98122a9ca4498bb89cd5c70007e702 ({username='oryx', principal='oryx/127.22.11.189@KRBTEST.COM'} at 127.0.0.1:33002)
I20260810 06:50:38.393090 713 tablet_copy_service.cc:161] P f1d212eb435b4fe99b8df6f621429257: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bb98122a9ca4498bb89cd5c70007e702 at {username='oryx', principal='oryx/127.22.11.189@KRBTEST.COM'} at 127.0.0.1:33002: session id = bb98122a9ca4498bb89cd5c70007e702-00000000000000000000000000000000
I20260810 06:50:38.397838 713 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:50:38.400441 1489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:50:38.407394 1489 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: tablet copy: Starting download of 0 data blocks...
I20260810 06:50:38.407752 1489 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: tablet copy: Starting download of 1 WAL segments...
I20260810 06:50:38.411319 1489 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:50:38.415884 713 tablet_copy_service.cc:342] P f1d212eb435b4fe99b8df6f621429257: Request end of tablet copy session bb98122a9ca4498bb89cd5c70007e702-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.11.189@KRBTEST.COM'} at 127.0.0.1:33002
I20260810 06:50:38.416078 713 tablet_copy_service.cc:433] P f1d212eb435b4fe99b8df6f621429257: ending tablet copy session bb98122a9ca4498bb89cd5c70007e702-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bb98122a9ca4498bb89cd5c70007e702
I20260810 06:50:38.427408 1489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:38.429802 1670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:38.435863 1674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:38.439569 1679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:38.440260 1489 server_base.cc:1034] running on GCE node
I20260810 06:50:38.440671 1489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:38.441324 1489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:38.442571 1489 hybrid_clock.cc:648] HybridClock initialized: now 1786344638442514 us; error 35 us; skew 500 ppm
I20260810 06:50:38.442787 1489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:38.444734 1489 init.cc:377] Logged in from keytab as oryx/127.22.11.189@KRBTEST.COM (short username oryx)
I20260810 06:50:38.445982 1489 webserver.cc:460] Webserver started at http://127.22.11.189:39827/ using document root <none> and password file <none>
I20260810 06:50:38.446308 1489 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:38.446380 1489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:38.448750 1489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:50:38.450515 1691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:38.450875 1489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260810 06:50:38.450999 1489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
uuid: "bb98122a9ca4498bb89cd5c70007e702"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "db978a8630540c515538985abfad9aaf"
server_key_iv: "5a1b3d862549c0f50f05326717887b4d"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.451149 1489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:38.469621 1489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:38.472785 1489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:38.491082 1489 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.189:41675
I20260810 06:50:38.491117 1775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.189:41675 every 8 connection(s)
I20260810 06:50:38.491973 1489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
I20260810 06:50:38.497691 1776 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:50:38.501865 1776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Bootstrap starting.
I20260810 06:50:38.516466 1776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:50:38.516952 1776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Bootstrap complete.
I20260810 06:50:38.517804 1776 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } }
I20260810 06:50:38.518015 1776 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bb98122a9ca4498bb89cd5c70007e702, State: Initialized, Role: LEARNER
I20260810 06:50:38.518249 1776 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } }
I20260810 06:50:38.519001 1780 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } }
I20260810 06:50:38.519304 1776 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:38.519380 1780 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:50:38.527045 1798 catalog_manager.cc:1259] Loaded cluster ID: 4d8b3452f15a4a3ea68023252ba6e2d8
I20260810 06:50:38.527127 1798 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: loading cluster ID for follower catalog manager: success
I20260810 06:50:38.529572 1798 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: acquiring CA information for follower catalog manager: success
I20260810 06:50:38.531385 1798 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:50:38.681456 1480 heartbeater.cc:503] Master 127.22.11.190:38069 was elected leader, sending a full tablet report...
I20260810 06:50:38.753600 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.744226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.0.0.1:52146)
0810 06:50:38.744482 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.744514 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.744687 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.745943 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.745948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.745994 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:38.746200 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.746212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.748610 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.748640 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.750629 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.750646 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.752761 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.752767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.753001 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.753065 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.753162 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.753303 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260810 06:50:38.753754 1800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.744478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.22.11.189:41675)
0810 06:50:38.745535 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.745541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.745566 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:50:38.745638 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.745644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.745733 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.745869 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:38.746359 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.748398 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.750796 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.752571 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.752725 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.752978 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.753204 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.753343 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":707,"threads_started":1}
I20260810 06:50:38.755232 1735 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260810 06:50:38.756002 1780 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d212eb435b4fe99b8df6f621429257. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } }
I20260810 06:50:38.756309 1780 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: LEARNER
W20260810 06:50:38.799474 1654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:38.799743 1654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:38.799885 1654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:38.807747 1654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:38.807917 1654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:38.808044 1654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:38.808161 1654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:38.808228 1654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:38.821587 1654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.188.keytab
--principal=oryx/127.22.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:42527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:38.823210 1654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:38.824777 1654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:38.839516 1806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:38.839874 1807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:38.839984 1809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:38.841701 1654 server_base.cc:1034] running on GCE node
I20260810 06:50:38.842594 1654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:38.844282 1654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:38.845556 1654 hybrid_clock.cc:648] HybridClock initialized: now 1786344638845409 us; error 80 us; skew 500 ppm
I20260810 06:50:38.846036 1654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:38.854481 1654 init.cc:377] Logged in from keytab as oryx/127.22.11.188@KRBTEST.COM (short username oryx)
I20260810 06:50:38.856429 1654 webserver.cc:460] Webserver started at http://127.22.11.188:41965/ using document root <none> and password file <none>
I20260810 06:50:38.857358 1654 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:38.857561 1654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:38.857893 1654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:38.861445 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/instance:
uuid: "f14f57b0e85e4c49898723fda3c31986"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "4e2948d222a925856f5fa07aaa15d366"
server_key_iv: "a9ec7968dac53d3610952aa8f928bb65"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.862351 1654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal/instance:
uuid: "f14f57b0e85e4c49898723fda3c31986"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "4e2948d222a925856f5fa07aaa15d366"
server_key_iv: "a9ec7968dac53d3610952aa8f928bb65"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.868992 1654 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 06:50:38.873446 1824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:38.874782 1654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:50:38.874949 1654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
uuid: "f14f57b0e85e4c49898723fda3c31986"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "4e2948d222a925856f5fa07aaa15d366"
server_key_iv: "a9ec7968dac53d3610952aa8f928bb65"
server_key_version: "encryptionkey@0"
I20260810 06:50:38.875214 1654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:38.891558 1654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:38.896792 1654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:38.897171 1654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:38.935859 1654 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.188:42527
I20260810 06:50:38.936041 1911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.188:42527 every 8 connection(s)
I20260810 06:50:38.937621 1654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
I20260810 06:50:38.942546 1912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:38.943045 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 1654
I20260810 06:50:38.951727 1912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.188@KRBTEST.COM
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.188@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:38.973181 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.954954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.22.11.190:38069)
0810 06:50:38.955752 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.955759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.956563 (+ 804us) server_negotiation.cc:407] Connection header received
0810 06:50:38.956855 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.956858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.956925 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.957010 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.958838 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.959811 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.961859 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.962599 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.962738 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.966410 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.966415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.966418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.966449 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.969979 (+ 3530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.970756 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.970760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.970762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.970907 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.971347 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.971351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.971354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.971686 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.971828 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.972410 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.972801 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":502,"threads_started":1}
I20260810 06:50:38.973658 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.945461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32894 (local address 127.22.11.188:42527)
0810 06:50:38.946458 (+ 997us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.946474 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.949166 (+ 2692us) server_negotiation.cc:407] Connection header received
0810 06:50:38.951828 (+ 2662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.951857 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.952408 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.952910 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.954442 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.957401 (+ 2959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.959093 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.959886 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.960162 (+ 276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.965953 (+ 5791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.965962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.965974 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.966003 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.969625 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.970259 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.970265 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.970267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.970422 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.970865 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.970870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.970873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.971512 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.971812 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.972141 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.972519 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":706,"thread_start_us":430,"threads_started":1}
I20260810 06:50:38.973698 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.955050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33008)
0810 06:50:38.956417 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.956458 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.956736 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.957122 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.957129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.958085 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.958674 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.958686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.959981 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.959986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.961702 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.961715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.962885 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.962892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.963041 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.963622 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.963643 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.966241 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.970122 (+ 3881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.970134 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.970145 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.970635 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.971018 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.971021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.971024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.971223 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.971814 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.971821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.972044 (+ 223us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.972273 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.972666 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":124,"spinlock_wait_cycles":15232,"thread_start_us":762,"threads_started":1}
I20260810 06:50:38.975428 1912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.188@KRBTEST.COM for oryx/127.22.11.189@KRBTEST.COM
I20260810 06:50:38.988282 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.976107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.0.0.1:52156)
0810 06:50:38.976250 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.976271 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.976360 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.976931 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.976934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.977437 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.977660 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.977671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.978566 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.978569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.979783 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.979796 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.980501 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.980505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.980767 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.981404 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.981420 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.983610 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.986496 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.986499 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.986502 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.986851 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.987141 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.987143 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.987145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.987260 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.987656 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.987659 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.987764 (+ 105us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.987859 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:50:38.988050 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260810 06:50:38.988333 1800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.976458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.22.11.189:41675)
0810 06:50:38.976679 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.976685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.976704 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:50:38.976755 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.976758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.976815 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.976886 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.977777 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.978442 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.979917 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.980386 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.980492 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.983786 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.983791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.983794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.983828 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.986404 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.986933 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.986936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.986938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.987056 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.987331 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.987334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.987336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.987577 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.987684 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:38.987927 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 06:50:38.988105 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:50:38.989185 1912 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:50:38 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344638, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.188@KRBTEST.COM for oryx/127.22.11.188@KRBTEST.COM
I20260810 06:50:39.000438 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.989688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.0.0.1:32904)
0810 06:50:38.989808 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:38.989821 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:38.989890 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:38.990148 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:38.990150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:38.990449 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:38.990587 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.990595 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.991524 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.991527 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.992319 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:38.992329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.993211 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.993215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:38.993429 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.993992 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:38.994014 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:38.995893 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:38.998525 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.998529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.998531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.998862 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.999156 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:38.999159 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:38.999162 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:38.999302 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:38.999722 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:38.999727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:38.999836 (+ 109us) client_negotiation.cc:769] Sending connection context
0810 06:50:38.999959 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.000156 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:50:39.000491 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:38.989834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32904 (local address 127.22.11.188:42527)
0810 06:50:38.989957 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:38.989961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:38.989974 (+ 13us) server_negotiation.cc:407] Connection header received
0810 06:50:38.990008 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:38.990010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:38.990057 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:38.990125 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:38.990697 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.991416 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.992428 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:38.993070 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:38.993177 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:38.996091 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:38.996096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:38.996100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:38.996126 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:38.998414 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.998955 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.998959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.998961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.999070 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:38.999385 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:38.999388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:38.999390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:38.999633 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:38.999732 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.000025 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.000266 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":43}
I20260810 06:50:39.005115 1912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Bootstrap starting.
I20260810 06:50:39.008312 1912 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:39.009482 1912 log.cc:824] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:39.011951 1912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: No bootstrap required, opened a new log
I20260810 06:50:39.018918 1912 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } }
I20260810 06:50:39.019232 1912 raft_consensus.cc:374] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:39.019378 1912 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14f57b0e85e4c49898723fda3c31986, State: Initialized, Role: FOLLOWER
I20260810 06:50:39.019922 1912 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } }
I20260810 06:50:39.020772 1925 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } } }
I20260810 06:50:39.021163 1925 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:50:39.021984 1912 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:39.028847 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 787
W20260810 06:50:39.032538 1945 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:50:39.032732 1945 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:50:39.036002 1654 master_runner.cc:428] Detected that this master f14f57b0e85e4c49898723fda3c31986 is joining an existing cluster
I20260810 06:50:39.036103 1654 master_runner.cc:432] Initiating AddMaster RPC to add 127.22.11.188:42527
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.129@KRBTEST.COM
I20260810 06:50:39.054390 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.030591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.22.11.129:33747)
0810 06:50:39.033360 (+ 2769us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.033368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.033398 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:50:39.033464 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.033471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.033673 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.033811 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.035346 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.037010 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.038934 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.039784 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.040513 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.045683 (+ 5170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.045697 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.045712 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.045756 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.050078 (+ 4322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.050958 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.050967 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.050970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.051131 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.051630 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.051635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.051639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.052085 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.052378 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.053711 (+ 1333us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.053981 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2574,"thread_start_us":2421,"threads_started":1}
I20260810 06:50:39.055989 1014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:39.068923 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.040051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.22.11.190:38069)
0810 06:50:39.040223 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.040228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.041331 (+ 1103us) server_negotiation.cc:407] Connection header received
0810 06:50:39.041617 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.041623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.041724 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.041799 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.043596 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.044713 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.057089 (+ 12376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.057934 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.058043 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.061429 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.061437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.061442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.061486 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.065443 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.066405 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.066411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.066416 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.066600 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.067102 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.067108 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.067111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.067489 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.067630 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.068258 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.068535 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:50:39.069612 1800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.039912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52170 (local address 127.22.11.189:41675)
0810 06:50:39.040076 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.040081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.049861 (+ 9780us) server_negotiation.cc:407] Connection header received
0810 06:50:39.050304 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.050310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.050402 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.050470 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.052073 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.053265 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.055388 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.056069 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.056165 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.061061 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.061068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.061073 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.061115 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.065423 (+ 4308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.066683 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.066689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.066692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.066866 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.067458 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.067463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.067467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.067805 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.067938 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.069036 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.069254 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:50:39.070822 1958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.039733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33014)
0810 06:50:39.041230 (+ 1497us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.041249 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.041448 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.042357 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.042362 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.043040 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.043409 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.043423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.044928 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.044933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.056878 (+ 11945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.056896 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.058458 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.058465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.058664 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.059489 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.059512 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.061122 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.065614 (+ 4492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.065620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.065624 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.066243 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.066756 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.066761 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.066765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.066970 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.067727 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.067732 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.067966 (+ 234us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.069421 (+ 1455us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.069987 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"thread_start_us":10149,"threads_started":1}
I20260810 06:50:39.071099 1957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.040438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.0.0.1:52170)
0810 06:50:39.049685 (+ 9247us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.049704 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.050150 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.050695 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.050700 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.051487 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.051879 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.051892 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.053472 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.053477 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.055203 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.055218 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.056475 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.056481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.057477 (+ 996us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.058195 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.058218 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.059464 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.065598 (+ 6134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.065603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.065607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.066536 (+ 929us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.067072 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.067077 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.067081 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.067318 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.068094 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.068098 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.068310 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.070118 (+ 1808us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.070441 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9104,"mutex_wait_us":721,"thread_start_us":479,"threads_started":1}
I20260810 06:50:39.089159 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.041600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32912 (local address 127.22.11.188:42527)
0810 06:50:39.041773 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.041778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.057239 (+ 15461us) server_negotiation.cc:407] Connection header received
0810 06:50:39.059777 (+ 2538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.059781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.059860 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.059937 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.070119 (+ 10182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.071802 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.075322 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.076360 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.076477 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.079925 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.079931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.079936 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.079987 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.085451 (+ 5464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.086402 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.086411 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.086415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.086570 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.087024 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.087029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.087033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.087554 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.087728 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.088314 (+ 586us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.088573 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:50:39.091051 1960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.041872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.0.0.1:32912)
0810 06:50:39.057053 (+ 15181us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.057071 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.059657 (+ 2586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.060129 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.060134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.066071 (+ 5937us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.068784 (+ 2713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.068797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.072658 (+ 3861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.072663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.075174 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.075190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.076898 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.076906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.077131 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.077799 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.077878 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.079739 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.085606 (+ 5867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.085610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.085615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.086276 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.086693 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.086697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.086705 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.086916 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.087799 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.087804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.087989 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.090018 (+ 2029us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.090338 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15040,"thread_start_us":14838,"threads_started":1}
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344636, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.190@KRBTEST.COM for oryx/127.22.11.188@KRBTEST.COM
I20260810 06:50:39.111191 1641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.077980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.0.0.1:32920)
0810 06:50:39.078320 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.078350 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.078454 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.089808 (+ 11354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.089813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.090474 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.090894 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.090912 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.093555 (+ 2643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.093562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.095528 (+ 1966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.095546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.097163 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.097170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.097389 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.098173 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.098198 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.102490 (+ 4292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.108314 (+ 5824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.108319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.108322 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.108863 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.109353 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.109359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.109362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.109521 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.110195 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.110204 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.110374 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.110524 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.110806 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260810 06:50:39.112382 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.079050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.22.11.188:42527)
0810 06:50:39.089385 (+ 10335us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.089390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.089406 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:50:39.089489 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.089494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.089668 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.089766 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.091122 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.093390 (+ 2268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.095739 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.096968 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.102669 (+ 5701us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.103007 (+ 338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.103014 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.103020 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.103055 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.108159 (+ 5104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.109061 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.109070 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.109073 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.109217 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.109674 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.109679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.109682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.110042 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.111662 (+ 1620us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.111781 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.112009 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10245,"thread_start_us":27936,"threads_started":1}
I20260810 06:50:39.114009 674 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.22.11.188:42527: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f14f57b0e85e4c49898723fda3c31986"
member_type: NON_VOTER
last_known_addr {
host: "127.22.11.188"
port: 42527
}
attrs {
promote: true
}
}
dest_uuid: "f1d212eb435b4fe99b8df6f621429257"
cas_config_opid_index: 5
I20260810 06:50:39.115120 674 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } }
I20260810 06:50:39.117630 1647 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f14f57b0e85e4c49898723fda3c31986 (127.22.11.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.125079 1735 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 LEARNER]: Refusing update from remote peer f1d212eb435b4fe99b8df6f621429257: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 06:50:39.127107 1866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:39.126595 1964 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260810 06:50:39.130002 1647 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.22.11.188:42527
I20260810 06:50:39.131026 1647 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.141345 1735 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f14f57b0e85e4c49898723fda3c31986 (127.22.11.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.144724 1647 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:39.146216 1780 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: NON_VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: true } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.146519 1780 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:50:39.149304 1964 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: attempting to promote NON_VOTER bb98122a9ca4498bb89cd5c70007e702 to VOTER
I20260810 06:50:39.150424 1964 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } }
I20260810 06:50:39.151254 1654 master.cc:537] Master@127.22.11.188:42527 shutting down...
W20260810 06:50:39.156069 1832 tablet.cc:2369] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:50:39.158666 1866 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Refusing update from remote peer f1d212eb435b4fe99b8df6f621429257: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 06:50:39.161895 1735 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 LEARNER]: Refusing update from remote peer f1d212eb435b4fe99b8df6f621429257: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 06:50:39.170356 1647 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 06:50:39.179013 1654 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:39.179486 1654 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:39.179764 1654 tablet_replica.cc:330] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: stopping tablet replica
W20260810 06:50:39.184034 649 consensus_peers.cc:487] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 -> Peer f14f57b0e85e4c49898723fda3c31986 (127.22.11.188:42527): Couldn't send request to peer f14f57b0e85e4c49898723fda3c31986. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 06:50:39.184700 1964 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bb98122a9ca4498bb89cd5c70007e702 (127.22.11.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.188187 1964 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.188429 1964 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:39.189236 1964 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.189517 1964 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:39.192657 1735 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bb98122a9ca4498bb89cd5c70007e702 (127.22.11.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.196484 1780 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.196735 1780 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:50:39.197980 1654 master.cc:552] Master@127.22.11.188:42527 shutdown complete.
I20260810 06:50:39.198068 1654 master_runner.cc:305] Clearing existing system tablet
I20260810 06:50:39.198493 1654 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:39.198642 1654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:39.205789 1654 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.000s
I20260810 06:50:39.208457 1977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:39.209069 1654 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:39.209334 1654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:50:39.209470 1654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
uuid: "f14f57b0e85e4c49898723fda3c31986"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "4e2948d222a925856f5fa07aaa15d366"
server_key_iv: "a9ec7968dac53d3610952aa8f928bb65"
server_key_version: "encryptionkey@0"
I20260810 06:50:39.214313 1654 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:39.222581 1654 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 06:50:39.222759 1654 log.cc:1196] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 06:50:39.223268 1654 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Deleting consensus metadata
I20260810 06:50:39.223578 1654 master_runner.cc:315] Copying system tablet from 127.22.11.190:38069
I20260810 06:50:39.225472 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 787
I20260810 06:50:39.236914 1654 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: tablet copy: Beginning tablet copy session from remote peer at address 127.22.11.190:38069
I20260810 06:50:39.251420 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:33747
--local_ip_for_outbound_sockets=127.22.11.129
--tserver_master_addrs=127.22.11.190:38069
--webserver_port=34277
--webserver_interface=127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab
--principal=oryx/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:50:39.254395 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.237785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.22.11.190:38069)
0810 06:50:39.238021 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.238027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.240037 (+ 2010us) server_negotiation.cc:407] Connection header received
0810 06:50:39.240268 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.240275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.240387 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.240505 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.241658 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.242779 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.244583 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.245342 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.245454 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.247757 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.247764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.247769 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.247809 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.251346 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.252152 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.252158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.252162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.252322 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.252842 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.252848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.252852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.253219 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.253378 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.253843 (+ 465us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.254066 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260810 06:50:39.254488 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.237737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33026)
0810 06:50:39.239902 (+ 2165us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.239929 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.240081 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.240589 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.240597 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.241180 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.241477 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.241490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.242946 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.242952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.244428 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.244448 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.245521 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.245527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.245731 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.246402 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.246426 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.247538 (+ 1112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.251533 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.251538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.251542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.252015 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.252462 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.252466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.252469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.252650 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.253347 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.253352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.253541 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.253712 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.254027 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2003,"thread_start_us":1872,"threads_started":1}
I20260810 06:50:39.255098 713 tablet_copy_service.cc:140] P f1d212eb435b4fe99b8df6f621429257: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f14f57b0e85e4c49898723fda3c31986 ({username='oryx', principal='oryx/127.22.11.188@KRBTEST.COM'} at 127.0.0.1:33026)
I20260810 06:50:39.255266 713 tablet_copy_service.cc:161] P f1d212eb435b4fe99b8df6f621429257: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f14f57b0e85e4c49898723fda3c31986 at {username='oryx', principal='oryx/127.22.11.188@KRBTEST.COM'} at 127.0.0.1:33026: session id = f14f57b0e85e4c49898723fda3c31986-00000000000000000000000000000000
I20260810 06:50:39.258558 713 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 06:50:39.260362 1654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 06:50:39.267176 1654 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: tablet copy: Starting download of 0 data blocks...
I20260810 06:50:39.267477 1654 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: tablet copy: Starting download of 1 WAL segments...
I20260810 06:50:39.270015 1654 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 06:50:39.274241 713 tablet_copy_service.cc:342] P f1d212eb435b4fe99b8df6f621429257: Request end of tablet copy session f14f57b0e85e4c49898723fda3c31986-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.22.11.188@KRBTEST.COM'} at 127.0.0.1:33026
I20260810 06:50:39.274389 713 tablet_copy_service.cc:433] P f1d212eb435b4fe99b8df6f621429257: ending tablet copy session f14f57b0e85e4c49898723fda3c31986-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f14f57b0e85e4c49898723fda3c31986
I20260810 06:50:39.286203 1654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:39.288805 1991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:39.294045 1995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:39.300814 1999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:39.301281 1654 server_base.cc:1034] running on GCE node
I20260810 06:50:39.301594 1654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:39.302201 1654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:39.303527 1654 hybrid_clock.cc:648] HybridClock initialized: now 1786344639303482 us; error 38 us; skew 500 ppm
I20260810 06:50:39.303762 1654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344639, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:39.305912 1654 init.cc:377] Logged in from keytab as oryx/127.22.11.188@KRBTEST.COM (short username oryx)
I20260810 06:50:39.313580 1654 webserver.cc:460] Webserver started at http://127.22.11.188:38661/ using document root <none> and password file <none>
I20260810 06:50:39.313947 1654 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:39.314059 1654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:39.317108 1654 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:50:39.319049 2012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:39.319523 1654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 06:50:39.319646 1654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
uuid: "f14f57b0e85e4c49898723fda3c31986"
format_stamp: "Formatted at 2026-08-10 06:50:38 on dist-test-slave-f7th"
server_key: "4e2948d222a925856f5fa07aaa15d366"
server_key_iv: "a9ec7968dac53d3610952aa8f928bb65"
server_key_version: "encryptionkey@0"
I20260810 06:50:39.319821 1654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:39.370959 1654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:39.374821 1654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:39.419620 1654 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.188:42527
I20260810 06:50:39.419642 2111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.188:42527 every 8 connection(s)
I20260810 06:50:39.421106 1654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
I20260810 06:50:39.433578 2112 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:50:39.437404 2112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Bootstrap starting.
I20260810 06:50:39.457417 2112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:50:39.457954 2112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: Bootstrap complete.
I20260810 06:50:39.459076 2112 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } }
I20260810 06:50:39.459390 2112 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f14f57b0e85e4c49898723fda3c31986, State: Initialized, Role: LEARNER
I20260810 06:50:39.459693 2112 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } }
I20260810 06:50:39.460996 2112 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:39.460717 2116 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.461320 2116 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:50:39.472244 2136 catalog_manager.cc:1259] Loaded cluster ID: 4d8b3452f15a4a3ea68023252ba6e2d8
I20260810 06:50:39.472355 2136 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: loading cluster ID for follower catalog manager: success
I20260810 06:50:39.476155 2136 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: acquiring CA information for follower catalog manager: success
I20260810 06:50:39.478922 2136 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:50:39.646026 1988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:39.646350 1988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:39.646533 1988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:39.656420 1988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:39.656533 1988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:39.656698 1988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:39.666682 1988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab
--principal=oryx/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:33747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=34277
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:39.668047 1988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:39.669360 1988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:39.681874 2143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:39.681803 2148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:39.681797 2145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:39.682549 1988 server_base.cc:1034] running on GCE node
I20260810 06:50:39.683161 1988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:39.684108 1988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:39.685319 1988 hybrid_clock.cc:648] HybridClock initialized: now 1786344639685261 us; error 51 us; skew 500 ppm
I20260810 06:50:39.685616 1988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344639, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:39.688648 1988 init.cc:377] Logged in from keytab as oryx/127.22.11.129@KRBTEST.COM (short username oryx)
I20260810 06:50:39.690186 1988 webserver.cc:460] Webserver started at http://127.22.11.129:34277/ using document root <none> and password file <none>
I20260810 06:50:39.690829 1988 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:39.690966 1988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:39.696029 1988 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:50:39.698714 2163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:39.700402 1988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:50:39.700584 1988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "7da9d94e1ade47dabb9975696e23b7c1"
format_stamp: "Formatted at 2026-08-10 06:50:36 on dist-test-slave-f7th"
server_key: "d19b75e58d5e341892a51ddcb00ccd06"
server_key_iv: "8b80fd5e38036b2a3d706fa739fd143c"
server_key_version: "encryptionkey@0"
I20260810 06:50:39.701689 1988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:39.732355 1988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:39.736070 2169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.730072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.0.0.1:32922)
0810 06:50:39.730656 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.730671 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.730780 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.731467 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.731473 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.731502 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:39.731717 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.731727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.733015 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.733020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.734157 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.734170 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.735286 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.735292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.735471 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.735509 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.735592 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.735728 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":376,"threads_started":1}
I20260810 06:50:39.736086 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.730222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.22.11.188:42527)
0810 06:50:39.731160 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.731164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.731178 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:50:39.731238 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.731241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.731313 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.731418 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:39.731865 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.732888 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.734317 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.735106 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.735232 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.735434 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.735638 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.735774 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":706,"threads_started":1}
I20260810 06:50:39.736492 1988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:39.736773 1988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:39.737001 2058 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 06:50:39.737516 2116 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d212eb435b4fe99b8df6f621429257. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:39.737818 2116 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: This master's current role is: LEARNER
I20260810 06:50:39.737881 1988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:39.739197 1988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:39.739293 1988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:39.739398 1988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:39.739478 1988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:39.801095 1988 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:33747
I20260810 06:50:39.801203 2354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:33747 every 8 connection(s)
I20260810 06:50:39.802524 1988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:39.808538 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 1988
I20260810 06:50:39.808928 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 1029
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344639, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.188@KRBTEST.COM
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344639, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:39.848100 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.816811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:36121 (local address 127.22.11.188:42527)
0810 06:50:39.820821 (+ 4010us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.820828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.820860 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:50:39.820933 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.820940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.821023 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.821134 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.823229 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.824242 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.827536 (+ 3294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.828379 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.828543 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.835599 (+ 7056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.835606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.835611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.835654 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.839774 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.845108 (+ 5334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.845115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.845119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.845312 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.845865 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.845870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.845873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.846250 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.846381 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.847472 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.847727 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3841}
I20260810 06:50:39.849494 2365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.806867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.22.11.129:36121)
0810 06:50:39.818122 (+ 11255us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.818184 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.818325 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.821448 (+ 3123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.821459 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.822368 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.823053 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.823072 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.824520 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.824526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.827379 (+ 2853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.827395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.829015 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.829027 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.829311 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.830193 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.830226 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.835324 (+ 5098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.844000 (+ 8676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.844010 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.844029 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.844912 (+ 883us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.845492 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.845497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.845500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.845733 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.846490 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.846498 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.847034 (+ 536us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.848653 (+ 1619us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.848930 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11112,"mutex_wait_us":49,"spinlock_wait_cycles":45824,"thread_start_us":13465,"threads_started":1}
I20260810 06:50:39.850178 2364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.806280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:38307)
0810 06:50:39.813576 (+ 7296us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.813624 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.816448 (+ 2824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.825835 (+ 9387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.825865 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.826450 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.826755 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.826768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.828871 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.828878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.830635 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.830656 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.833979 (+ 3323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.833986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.834212 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.834877 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.834902 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.837761 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.844608 (+ 6847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.844630 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.844634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.845204 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.845695 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.845701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.845705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.845875 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.846582 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.846587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.847006 (+ 419us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.847332 (+ 326us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.847926 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6941,"mutex_wait_us":3,"thread_start_us":6597,"threads_started":1}
I20260810 06:50:39.850224 2362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.805595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:38307 (local address 127.22.11.190:38069)
0810 06:50:39.825504 (+ 19909us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.825510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.825532 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:50:39.825592 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.825597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.825673 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.825787 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.827583 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.828714 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.832519 (+ 3805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.833756 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.833886 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.840039 (+ 6153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.840045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.840050 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.840091 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.843791 (+ 3700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.845391 (+ 1600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.845397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.845400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.845561 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.846002 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.846007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.846011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.846371 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.846506 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.849572 (+ 3066us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.849822 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18611,"thread_start_us":18671,"threads_started":1}
I20260810 06:50:39.851881 2356 heartbeater.cc:347] Connected to a master server at 127.22.11.190:38069
I20260810 06:50:39.852250 2356 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:39.853330 2356 heartbeater.cc:511] Master 127.22.11.190:38069 requested a full tablet report, sending...
I20260810 06:50:39.854218 2355 heartbeater.cc:347] Connected to a master server at 127.22.11.188:42527
I20260810 06:50:39.854352 2355 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:39.854826 2355 heartbeater.cc:511] Master 127.22.11.188:42527 requested a full tablet report, sending...
I20260810 06:50:39.856187 674 ts_manager.cc:194] Re-registered known tserver with Master: 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:39.857120 2046 ts_manager.cc:194] Registered new tserver with Master: 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:39.857267 674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:38307
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344639, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.189@KRBTEST.COM
I20260810 06:50:39.866232 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.820988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:42395 (local address 127.22.11.189:41675)
0810 06:50:39.848676 (+ 27688us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.848682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.848708 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:50:39.848767 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.848772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.848872 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.848990 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.851923 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.852841 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.855096 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.855688 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.855811 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.860292 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.860299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.860304 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.860346 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.863673 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.864257 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.864263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.864266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.864415 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.864865 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.864869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.864871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.865140 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.865272 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.865647 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.865842 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27549,"thread_start_us":28159,"threads_started":1}
I20260810 06:50:39.866284 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.830897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.22.11.129:42395)
0810 06:50:39.847072 (+ 16175us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:39.847092 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:39.847212 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:39.849318 (+ 2106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:39.849323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:39.849922 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:39.850270 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.850286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.852992 (+ 2706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.852998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.854821 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:39.854837 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.856085 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.856091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:39.856337 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.856984 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:39.857008 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:39.860060 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:39.863807 (+ 3747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.863812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.863816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.864148 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.864537 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:39.864541 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:39.864543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:39.864736 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:39.865262 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:39.865267 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:39.865404 (+ 137us) client_negotiation.cc:769] Sending connection context
0810 06:50:39.865527 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:50:39.865821 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16030,"thread_start_us":16393,"threads_started":1}
Aug 10 06:50:39 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.130@KRBTEST.COM
I20260810 06:50:39.867611 2360 heartbeater.cc:347] Connected to a master server at 127.22.11.189:41675
I20260810 06:50:39.867743 2360 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:39.868256 2360 heartbeater.cc:511] Master 127.22.11.189:41675 requested a full tablet report, sending...
I20260810 06:50:39.870100 1717 ts_manager.cc:194] Registered new tserver with Master: 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:39.876368 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:39.817607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.22.11.130:33987)
0810 06:50:39.853419 (+ 35812us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:39.853427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:39.854430 (+ 1003us) server_negotiation.cc:407] Connection header received
0810 06:50:39.854520 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:39.854529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:39.854766 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:39.855079 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:39.857632 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.859245 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.861092 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:39.862046 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:39.862179 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:39.867999 (+ 5820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:39.868013 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:39.868031 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:39.868081 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:39.872861 (+ 4780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.873761 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.873769 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.873773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.874019 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:39.874383 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:39.874388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:39.874391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:39.874865 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:39.875275 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:39.875453 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 06:50:39.875770 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35626,"thread_start_us":39019,"threads_started":1}
I20260810 06:50:39.878304 1227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:39.983464 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 1029
I20260810 06:50:40.009936 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:33987
--local_ip_for_outbound_sockets=127.22.11.130
--tserver_master_addrs=127.22.11.190:38069
--webserver_port=45699
--webserver_interface=127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab
--principal=oryx/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:50:40.183393 1647 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: NON_VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: true } } }
I20260810 06:50:40.183917 1647 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:40.185636 2382 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: attempting to promote NON_VOTER f14f57b0e85e4c49898723fda3c31986 to VOTER
I20260810 06:50:40.186508 2382 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:40.188728 2058 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 LEARNER]: Refusing update from remote peer f1d212eb435b4fe99b8df6f621429257: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 06:50:40.188906 1735 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Refusing update from remote peer f1d212eb435b4fe99b8df6f621429257: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 06:50:40.189380 1647 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 06:50:40.189739 2381 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 06:50:40.192476 2381 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f14f57b0e85e4c49898723fda3c31986 (127.22.11.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.192904 1735 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f14f57b0e85e4c49898723fda3c31986 (127.22.11.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.197075 2381 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.197398 2381 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:40.197157 2385 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.197702 2385 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:40.197642 2381 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.197834 2381 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:40.197726 2058 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f14f57b0e85e4c49898723fda3c31986 (127.22.11.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.201089 2386 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.201483 2386 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:50:40.202203 2116 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1d212eb435b4fe99b8df6f621429257" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:40.202457 2116 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 06:50:40.343981 2380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:40.344223 2380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:40.344324 2380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:40.351414 2380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:40.351513 2380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:40.351634 2380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:40.360920 2380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab
--principal=oryx/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:33987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=45699
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:40.362130 2380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:40.363322 2380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:40.375774 2402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:40.375892 2395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:40.375919 2399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:40.376745 2380 server_base.cc:1034] running on GCE node
I20260810 06:50:40.377334 2380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:40.378255 2380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:40.379463 2380 hybrid_clock.cc:648] HybridClock initialized: now 1786344640379410 us; error 43 us; skew 500 ppm
I20260810 06:50:40.379752 2380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:40 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:40.382838 2380 init.cc:377] Logged in from keytab as oryx/127.22.11.130@KRBTEST.COM (short username oryx)
I20260810 06:50:40.384094 2380 webserver.cc:460] Webserver started at http://127.22.11.130:45699/ using document root <none> and password file <none>
I20260810 06:50:40.384744 2380 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:40.384863 2380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:40.389729 2380 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:50:40.392148 2412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:40.393421 2380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:50:40.393584 2380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "b74fa238fb9948a580de569f784c290f"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "db7c902160f4e8c50b2acefc24563f6e"
server_key_iv: "d7ce59a54c828f160a285f528b296fab"
server_key_version: "encryptionkey@0"
I20260810 06:50:40.394482 2380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:40.413607 2380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:40.417572 2380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:40.417834 2380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:40.418836 2380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:40.420671 2380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:40.420786 2380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:50:40.420909 2380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:40.420994 2380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:40.454931 2380 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:33987
I20260810 06:50:40.454988 2607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:33987 every 8 connection(s)
I20260810 06:50:40.456318 2380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:40.463376 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 2380
I20260810 06:50:40.463819 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 1250
Aug 10 06:50:40 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:40.492729 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:40.459105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:37757 (local address 127.22.11.190:38069)
0810 06:50:40.459564 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:40.459570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:40.461985 (+ 2415us) server_negotiation.cc:407] Connection header received
0810 06:50:40.467264 (+ 5279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:40.467271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:40.467378 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:40.467459 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:40.471670 (+ 4211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.473154 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.476611 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.477599 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.477706 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:40.483880 (+ 6174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:40.483887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:40.483892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:40.483937 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:40.488243 (+ 4306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.489512 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.489518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.489521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.489731 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.490475 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.490480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.490484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.490929 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:40.491063 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:40.492086 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0810 06:50:40.492327 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":550,"threads_started":1}
Aug 10 06:50:40 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.131@KRBTEST.COM
I20260810 06:50:40.495342 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:40.459631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:37757)
0810 06:50:40.461417 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:40.461470 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:40.466926 (+ 5456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:40.468127 (+ 1201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:40.468138 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:40.469281 (+ 1143us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:40.470561 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:40.470617 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.473439 (+ 2822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.473445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:40.476417 (+ 2972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:40.476436 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.478316 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.478324 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:40.478746 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:40.479603 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:40.479629 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:40.483506 (+ 3877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:40.488422 (+ 4916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:40.488430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:40.488448 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:40.489350 (+ 902us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:40.489919 (+ 569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:40.489924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:40.489927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:40.490295 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:40.491153 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:40.491162 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:40.491616 (+ 454us) client_negotiation.cc:769] Sending connection context
0810 06:50:40.492960 (+ 1344us) client_negotiation.cc:240] Negotiation successful
0810 06:50:40.493503 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"spinlock_wait_cycles":13440,"thread_start_us":24718,"threads_started":1}
I20260810 06:50:40.497426 2614 heartbeater.cc:347] Connected to a master server at 127.22.11.190:38069
I20260810 06:50:40.497853 2614 heartbeater.cc:464] Registering TS with master...
Aug 10 06:50:40 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.188@KRBTEST.COM
I20260810 06:50:40.499078 2614 heartbeater.cc:511] Master 127.22.11.190:38069 requested a full tablet report, sending...
I20260810 06:50:40.500895 674 ts_manager.cc:194] Re-registered known tserver with Master: b74fa238fb9948a580de569f784c290f (127.22.11.130:33987)
I20260810 06:50:40.502210 674 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:37757
I20260810 06:50:40.504765 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:40.465472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.22.11.131:43297)
0810 06:50:40.475026 (+ 9554us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:40.475037 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:40.475892 (+ 855us) server_negotiation.cc:407] Connection header received
0810 06:50:40.476050 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:40.476061 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:40.476314 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:40.476461 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:40.480891 (+ 4430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.483499 (+ 2608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.485953 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.487197 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.487338 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:40.495005 (+ 7667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:40.495020 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:40.495049 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:40.495092 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:40.500047 (+ 4955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.501027 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.501037 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.501040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.501474 (+ 434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.501985 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.501991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.501995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.502686 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:40.503061 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:40.503795 (+ 734us) server_negotiation.cc:299] Negotiation successful
0810 06:50:40.504127 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9301,"thread_start_us":9051,"threads_started":1}
I20260810 06:50:40.506515 1471 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 10 06:50:40 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.189@KRBTEST.COM
I20260810 06:50:40.511693 2625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:40.462228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:47627 (local address 127.22.11.188:42527)
0810 06:50:40.471540 (+ 9312us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:40.471546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:40.484821 (+ 13275us) server_negotiation.cc:407] Connection header received
0810 06:50:40.484902 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:40.484907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:40.484986 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:40.485086 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:40.489990 (+ 4904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.491653 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.493960 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.494842 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.494937 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:40.500015 (+ 5078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:40.500024 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:40.500029 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:40.500076 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:40.504760 (+ 4684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.507884 (+ 3124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.507891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.507894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.508132 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.509205 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.509211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.509214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.509592 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:40.509727 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:40.510354 (+ 627us) server_negotiation.cc:299] Negotiation successful
0810 06:50:40.510822 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9167,"thread_start_us":8942,"threads_started":1}
I20260810 06:50:40.512812 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:40.461728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.22.11.130:47627)
0810 06:50:40.484217 (+ 22489us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:40.484237 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:40.484377 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:40.485226 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:40.485230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:40.488954 (+ 3724us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:40.489798 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:40.489813 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.491862 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.491867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:40.493785 (+ 1918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:40.493799 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.495409 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.495416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:40.495602 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:40.496202 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:40.496223 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:40.499684 (+ 3461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:40.506249 (+ 6565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:40.506256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:40.506259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:40.507733 (+ 1474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:40.508287 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:40.508292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:40.508295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:40.508975 (+ 680us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:40.509809 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:40.509813 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:40.510014 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 06:50:40.511960 (+ 1946us) client_negotiation.cc:240] Negotiation successful
0810 06:50:40.512260 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22347,"mutex_wait_us":2,"thread_start_us":16732,"threads_started":1}
I20260810 06:50:40.513955 2609 heartbeater.cc:347] Connected to a master server at 127.22.11.188:42527
I20260810 06:50:40.514168 2609 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:40.514755 2609 heartbeater.cc:511] Master 127.22.11.188:42527 requested a full tablet report, sending...
I20260810 06:50:40.516031 2046 ts_manager.cc:194] Registered new tserver with Master: b74fa238fb9948a580de569f784c290f (127.22.11.130:33987)
I20260810 06:50:40.518918 2618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:40.459102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:43509 (local address 127.22.11.189:41675)
0810 06:50:40.471470 (+ 12368us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:40.471477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:40.481137 (+ 9660us) server_negotiation.cc:407] Connection header received
0810 06:50:40.481218 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:40.481223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:40.481368 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:40.481478 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:40.483480 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.484994 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.487545 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.488392 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.488504 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:40.508553 (+ 20049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:40.508560 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:40.508565 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:40.508652 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:40.514272 (+ 5620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.515371 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.515377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.515382 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.515628 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:40.516083 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:40.516088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:40.516092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:40.516519 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:40.516737 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:40.518288 (+ 1551us) server_negotiation.cc:299] Negotiation successful
0810 06:50:40.518523 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12237,"thread_start_us":12367,"threads_started":1}
I20260810 06:50:40.519145 2623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:40.459631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.22.11.130:43509)
0810 06:50:40.480851 (+ 21220us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:40.480873 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:40.481013 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:40.481667 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:40.481672 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:40.482698 (+ 1026us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:40.483268 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:40.483284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.485456 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.485461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:40.487389 (+ 1928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:40.487403 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:40.488930 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:40.488940 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:40.489215 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:40.504847 (+ 15632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:40.504872 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:40.508316 (+ 3444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:40.514418 (+ 6102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:40.514423 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:40.514427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:40.515206 (+ 779us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:40.515757 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:40.515762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:40.515765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:40.515959 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:40.516674 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:40.516680 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:40.516970 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 06:50:40.517120 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 06:50:40.517458 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21023,"mutex_wait_us":89,"spinlock_wait_cycles":52864,"thread_start_us":963,"threads_started":1}
I20260810 06:50:40.520059 2615 heartbeater.cc:347] Connected to a master server at 127.22.11.189:41675
I20260810 06:50:40.520211 2615 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:40.520943 2615 heartbeater.cc:511] Master 127.22.11.189:41675 requested a full tablet report, sending...
I20260810 06:50:40.522199 1717 ts_manager.cc:194] Registered new tserver with Master: b74fa238fb9948a580de569f784c290f (127.22.11.130:33987)
I20260810 06:50:40.599646 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 1250
I20260810 06:50:40.626236 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:43297
--local_ip_for_outbound_sockets=127.22.11.131
--tserver_master_addrs=127.22.11.190:38069
--webserver_port=40425
--webserver_interface=127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab
--principal=oryx/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:50:40.860061 2356 heartbeater.cc:503] Master 127.22.11.190:38069 was elected leader, sending a full tablet report...
W20260810 06:50:40.957875 2631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:40.958141 2631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:40.958282 2631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:40.965444 2631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:40.965550 2631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:40.965715 2631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:40.975041 2631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab
--principal=oryx/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:43297
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=40425
--tserver_master_addrs=127.22.11.190:38069,127.22.11.189:41675,127.22.11.188:42527
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:40.976223 2631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:40.977502 2631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:40.989914 2640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:40.989953 2637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:40.989982 2646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:40.990568 2631 server_base.cc:1034] running on GCE node
I20260810 06:50:40.991166 2631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:40.992015 2631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:40.993202 2631 hybrid_clock.cc:648] HybridClock initialized: now 1786344640993165 us; error 27 us; skew 500 ppm
I20260810 06:50:40.993499 2631 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:40 dist-test-slave-f7th krb5kdc[603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:40.996414 2631 init.cc:377] Logged in from keytab as oryx/127.22.11.131@KRBTEST.COM (short username oryx)
I20260810 06:50:40.997840 2631 webserver.cc:460] Webserver started at http://127.22.11.131:40425/ using document root <none> and password file <none>
I20260810 06:50:40.998483 2631 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:40.998622 2631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:41.003767 2631 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:50:41.006489 2655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:41.007784 2631 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:41.007941 2631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "36b16974d96a42b7bf934f165eef02c2"
format_stamp: "Formatted at 2026-08-10 06:50:37 on dist-test-slave-f7th"
server_key: "21561ce9f76abea2f81e3fd44d80a27b"
server_key_iv: "3cae663d334e5ffe6ee5ac243460a2d3"
server_key_version: "encryptionkey@0"
I20260810 06:50:41.008989 2631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:41.021773 2631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:41.025482 2631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:41.025700 2631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:41.026556 2631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:41.027832 2631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:41.027917 2631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:41.027994 2631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:41.028047 2631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:41.058281 2631 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:43297
I20260810 06:50:41.058332 2851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:43297 every 8 connection(s)
I20260810 06:50:41.059676 2631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:41.069306 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 2631
I20260810 06:50:41.115144 2875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.079076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51314 (local address 127.22.11.189:41675)
0810 06:50:41.091876 (+ 12800us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.091881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.093936 (+ 2055us) server_negotiation.cc:407] Connection header received
0810 06:50:41.094014 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.094020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.094105 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.094409 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.096900 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.098114 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.100090 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.100796 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.100904 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.104787 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.104793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.104798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.104862 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.110245 (+ 5383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.111332 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.111338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.111342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.111575 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.112163 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.112168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.112172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.112608 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.112976 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.114321 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.114603 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12680,"thread_start_us":12588,"threads_started":1}
Aug 10 06:50:41 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.189@KRBTEST.COM
Aug 10 06:50:41 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.188@KRBTEST.COM
Aug 10 06:50:41 dist-test-slave-f7th krb5kdc[603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344640, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:41.138494 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.095620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60630 (local address 127.22.11.190:38069)
0810 06:50:41.100170 (+ 4550us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.100176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.100192 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:50:41.100244 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.100249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.100316 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.104573 (+ 4257us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.112811 (+ 8238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.120046 (+ 7235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.122594 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.123454 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.124244 (+ 790us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.128454 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.128461 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.128466 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.128512 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.133795 (+ 5283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.135022 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.135028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.135031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.135250 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.135804 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.135810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.135813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.136148 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.137346 (+ 1198us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.137571 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.137841 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4434,"thread_start_us":5067,"threads_started":1}
I20260810 06:50:41.147806 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.091842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:49177 (local address 127.22.11.189:41675)
0810 06:50:41.093016 (+ 1174us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.093022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.119933 (+ 26911us) server_negotiation.cc:407] Connection header received
0810 06:50:41.120148 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.120152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.120270 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.120353 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.124787 (+ 4434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.126429 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.129759 (+ 3330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.130545 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.130639 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.137630 (+ 6991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.137638 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.137642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.137711 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.142793 (+ 5082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.143945 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.143950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.143953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.144189 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.144996 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.145001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.145005 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.145513 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.145719 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.146914 (+ 1195us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.147178 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1033,"thread_start_us":933,"threads_started":1}
I20260810 06:50:41.149068 2864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.063397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:47477 (local address 127.22.11.190:38069)
0810 06:50:41.094860 (+ 31463us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.094866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.094888 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:50:41.094953 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.094959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.095050 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.095328 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.114191 (+ 18863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.115970 (+ 1779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.127605 (+ 11635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.128591 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.128741 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.138496 (+ 9755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.138503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.138507 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.138544 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.144819 (+ 6275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.145850 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.145855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.145859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.146048 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.146618 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.146623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.146630 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.147079 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.147223 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.148187 (+ 964us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.148408 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31332,"thread_start_us":32062,"threads_started":1}
I20260810 06:50:41.150321 2885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.093718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44924 (local address 127.22.11.188:42527)
0810 06:50:41.116544 (+ 22826us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.116550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.116595 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:50:41.116676 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.116681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.116976 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.117089 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.120275 (+ 3186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.121666 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.125523 (+ 3857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.126504 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.126622 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.132096 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.132104 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.132109 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.132147 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.144510 (+ 12363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.145635 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.145640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.145644 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.145867 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.146441 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.146447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.146450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.146948 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.147182 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.149636 (+ 2454us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.149881 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22667,"thread_start_us":23078,"threads_started":1}
I20260810 06:50:41.150730 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.091432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.22.11.131:49177)
0810 06:50:41.119829 (+ 28397us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.119852 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.120026 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.122490 (+ 2464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.122493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.123460 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:41.124368 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.124384 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.126600 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.126606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.129602 (+ 2996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.129617 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.131094 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.131102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.131656 (+ 554us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.132572 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:41.132593 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:41.137014 (+ 4421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:41.142979 (+ 5965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:41.142988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:41.143006 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:41.143780 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:41.144388 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:41.144393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:41.144396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:41.144845 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:41.145826 (+ 981us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:41.145844 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:41.146346 (+ 502us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.148462 (+ 2116us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.148967 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28221,"mutex_wait_us":195,"thread_start_us":5021,"threads_started":1}
I20260810 06:50:41.151594 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.063251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:47477)
0810 06:50:41.087004 (+ 23753us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.087067 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.089979 (+ 2912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.100839 (+ 10860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.100845 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.101349 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:41.113940 (+ 12591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.113959 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.116263 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.116269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.127442 (+ 11173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.127457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.131258 (+ 3801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.131266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.131545 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.133028 (+ 1483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:41.133051 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:41.138231 (+ 5180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:41.144981 (+ 6750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:41.144985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:41.144989 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:41.145694 (+ 705us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:41.146189 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:41.146194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:41.146197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:41.146475 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:41.147302 (+ 827us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:41.147307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:41.147536 (+ 229us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.147724 (+ 188us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.149798 (+ 2074us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23366,"mutex_wait_us":3,"thread_start_us":56039,"threads_started":1}
I20260810 06:50:41.152571 2860 heartbeater.cc:347] Connected to a master server at 127.22.11.189:41675
I20260810 06:50:41.152969 2860 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:41.154184 2860 heartbeater.cc:511] Master 127.22.11.189:41675 requested a full tablet report, sending...
I20260810 06:50:41.155104 2859 heartbeater.cc:347] Connected to a master server at 127.22.11.190:38069
I20260810 06:50:41.155277 2859 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:41.156188 2859 heartbeater.cc:511] Master 127.22.11.190:38069 requested a full tablet report, sending...
I20260810 06:50:41.156286 1717 ts_manager.cc:194] Registered new tserver with Master: 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297)
I20260810 06:50:41.156972 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.062566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:50703 (local address 127.22.11.188:42527)
0810 06:50:41.072991 (+ 10425us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.072996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.096429 (+ 23433us) server_negotiation.cc:407] Connection header received
0810 06:50:41.096855 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.096861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.096942 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.097192 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.121830 (+ 24638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.123603 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.128272 (+ 4669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.129307 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.129404 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.138676 (+ 9272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.138684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.138688 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.138729 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.152259 (+ 13530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.153317 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.153323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.153326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.153533 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.154126 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.154131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.154134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.154528 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.154710 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.156042 (+ 1332us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.156294 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10326,"thread_start_us":10187,"threads_started":1}
I20260810 06:50:41.157454 2883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.063570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.22.11.131:50703)
0810 06:50:41.096212 (+ 32642us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.096238 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.096700 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.097731 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.097745 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.098743 (+ 998us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:41.099692 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.099717 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.123831 (+ 24114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.123837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.128094 (+ 4257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.128108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.130191 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.130200 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.130556 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.132124 (+ 1568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:41.132158 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:41.137290 (+ 5132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:41.152395 (+ 15105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:41.152400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:41.152404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:41.153171 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:41.153696 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:41.153701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:41.153709 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:41.153991 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:41.154678 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:41.154684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:41.154925 (+ 241us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.155139 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.155408 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32507,"thread_start_us":26837,"threads_started":1}
I20260810 06:50:41.157963 671 ts_manager.cc:194] Re-registered known tserver with Master: 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297)
I20260810 06:50:41.145387 674 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 06:50:41.159231 671 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:47477
I20260810 06:50:41.160645 2856 heartbeater.cc:347] Connected to a master server at 127.22.11.188:42527
I20260810 06:50:41.160848 2856 heartbeater.cc:464] Registering TS with master...
W20260810 06:50:41.160856 674 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:41.161701 2856 heartbeater.cc:511] Master 127.22.11.188:42527 requested a full tablet report, sending...
I20260810 06:50:41.162808 2046 ts_manager.cc:194] Registered new tserver with Master: 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297)
I20260810 06:50:41.208138 2901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.185948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40566 (local address 127.22.11.131:43297)
0810 06:50:41.198248 (+ 12300us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.198256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.198293 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:50:41.198414 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.198423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.199618 (+ 1195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.200824 (+ 1206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:41.201425 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.203068 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.205480 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.206993 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.207108 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.207320 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.207590 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.207752 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12118,"thread_start_us":11912,"threads_started":1}
I20260810 06:50:41.208154 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.186734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:33747 (local address 127.0.0.1:38606)
0810 06:50:41.188083 (+ 1349us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.188107 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.188221 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.202155 (+ 13934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.202160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.202186 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:41.202827 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.202841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.204325 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.204329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.205937 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.205951 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.207269 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.207275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.207452 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.207478 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.207530 (+ 52us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.207700 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1232,"mutex_wait_us":39,"thread_start_us":11843,"threads_started":1}
I20260810 06:50:41.208536 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.187331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:33987 (local address 127.0.0.1:57862)
0810 06:50:41.198455 (+ 11124us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.198469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.198569 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.199190 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.199193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.199215 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:41.199588 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.199601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.201624 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.201630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.204595 (+ 2965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.204634 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.207695 (+ 3061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.207701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.207865 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.207909 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.208004 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.208124 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11020,"mutex_wait_us":3,"thread_start_us":1314,"threads_started":1}
I20260810 06:50:41.209194 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.185525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:43297 (local address 127.0.0.1:40566)
0810 06:50:41.187054 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.187075 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.187226 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.200828 (+ 13602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.200832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.200859 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:41.201176 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.201189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.203221 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.203227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.205349 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.205365 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.207184 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.207191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.207395 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.207431 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.207529 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.208925 (+ 1396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1231,"mutex_wait_us":1316,"thread_start_us":1463,"threads_started":1}
I20260810 06:50:41.209213 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.186685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38606 (local address 127.22.11.129:33747)
0810 06:50:41.201459 (+ 14774us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.201468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.201494 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:50:41.201589 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.201597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.201816 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.203023 (+ 1207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:41.203278 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.204186 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.206102 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.207043 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.208551 (+ 1508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.208794 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.208882 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.209009 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14599,"thread_start_us":14447,"threads_started":1}
I20260810 06:50:41.209758 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.187888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57862 (local address 127.22.11.130:33987)
0810 06:50:41.189682 (+ 1794us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.189690 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.198556 (+ 8866us) server_negotiation.cc:407] Connection header received
0810 06:50:41.198689 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.198697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.198944 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.199211 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:41.199738 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.201442 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.205541 (+ 4099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.206974 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.207108 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.207313 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.209410 (+ 2097us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.209573 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1649,"thread_start_us":1528,"threads_started":1}
I20260810 06:50:41.213171 2746 tablet_service.cc:1467] Processing CreateTablet for tablet bd0e238a82fc4464977296f738fe59d7 (DEFAULT_TABLE table=test-table [id=63120499b9fc4cfcbc631caf9e431998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:41.213169 2254 tablet_service.cc:1467] Processing CreateTablet for tablet bd0e238a82fc4464977296f738fe59d7 (DEFAULT_TABLE table=test-table [id=63120499b9fc4cfcbc631caf9e431998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:41.213168 2498 tablet_service.cc:1467] Processing CreateTablet for tablet bd0e238a82fc4464977296f738fe59d7 (DEFAULT_TABLE table=test-table [id=63120499b9fc4cfcbc631caf9e431998]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:41.214454 2746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd0e238a82fc4464977296f738fe59d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:41.214506 2498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd0e238a82fc4464977296f738fe59d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:41.214534 2254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd0e238a82fc4464977296f738fe59d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:41.227785 2918 tablet_bootstrap.cc:492] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Bootstrap starting.
I20260810 06:50:41.231685 2919 tablet_bootstrap.cc:492] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Bootstrap starting.
I20260810 06:50:41.233211 2918 tablet_bootstrap.cc:654] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:41.235368 2918 log.cc:824] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:41.237023 2919 tablet_bootstrap.cc:654] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:41.237988 2920 tablet_bootstrap.cc:492] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Bootstrap starting.
I20260810 06:50:41.238889 2919 log.cc:824] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:41.241716 2920 tablet_bootstrap.cc:654] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:41.242571 2919 tablet_bootstrap.cc:492] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: No bootstrap required, opened a new log
I20260810 06:50:41.242903 2919 ts_tablet_manager.cc:1397] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260810 06:50:41.242916 2920 log.cc:824] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:41.245030 2918 tablet_bootstrap.cc:492] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: No bootstrap required, opened a new log
I20260810 06:50:41.245301 2918 ts_tablet_manager.cc:1397] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.008s
I20260810 06:50:41.245604 2920 tablet_bootstrap.cc:492] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: No bootstrap required, opened a new log
I20260810 06:50:41.245843 2920 ts_tablet_manager.cc:1397] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:50:41.254737 2919 raft_consensus.cc:348] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.255368 2919 raft_consensus.cc:374] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:41.255517 2919 raft_consensus.cc:729] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74fa238fb9948a580de569f784c290f, State: Initialized, Role: FOLLOWER
I20260810 06:50:41.256443 2919 consensus_queue.cc:260] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.257527 2918 raft_consensus.cc:348] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.258025 2918 raft_consensus.cc:374] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:41.258126 2918 raft_consensus.cc:729] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b16974d96a42b7bf934f165eef02c2, State: Initialized, Role: FOLLOWER
I20260810 06:50:41.258298 2920 raft_consensus.cc:348] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.258772 2920 raft_consensus.cc:374] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:41.258806 2919 ts_tablet_manager.cc:1428] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Time spent starting tablet: real 0.016s user 0.001s sys 0.013s
I20260810 06:50:41.258924 2920 raft_consensus.cc:729] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7da9d94e1ade47dabb9975696e23b7c1, State: Initialized, Role: FOLLOWER
I20260810 06:50:41.259217 2614 heartbeater.cc:503] Master 127.22.11.190:38069 was elected leader, sending a full tablet report...
I20260810 06:50:41.259639 2920 consensus_queue.cc:260] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.258855 2918 consensus_queue.cc:260] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.263477 2920 ts_tablet_manager.cc:1428] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
I20260810 06:50:41.268941 2859 heartbeater.cc:503] Master 127.22.11.190:38069 was elected leader, sending a full tablet report...
I20260810 06:50:41.269374 2918 ts_tablet_manager.cc:1428] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Time spent starting tablet: real 0.024s user 0.013s sys 0.005s
W20260810 06:50:41.306382 2361 tablet.cc:2369] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:50:41.319548 2861 tablet.cc:2369] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:50:41.427234 2927 raft_consensus.cc:482] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:41.427886 2927 raft_consensus.cc:504] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.430603 2927 leader_election.cc:288] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297), 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:41.441574 2908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.431075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:54429 (local address 127.22.11.129:33747)
0810 06:50:41.431297 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.431303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.433956 (+ 2653us) server_negotiation.cc:407] Connection header received
0810 06:50:41.434350 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.434355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.434438 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.434540 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:41.435546 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.436685 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.439224 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.440459 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.440602 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.440705 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.441137 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.441262 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:50:41.441849 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.431454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:33747 (local address 127.22.11.130:54429)
0810 06:50:41.433821 (+ 2367us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.433843 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.434151 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.434563 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.434567 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.434590 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:41.435342 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.435355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.436833 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.436837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.439042 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.439055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.440760 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.440766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.440913 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.440938 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.441100 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.441229 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2071,"thread_start_us":2501,"threads_started":1}
I20260810 06:50:41.442186 2901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.431063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:55921 (local address 127.22.11.131:43297)
0810 06:50:41.431335 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.431341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.434493 (+ 3152us) server_negotiation.cc:407] Connection header received
0810 06:50:41.434730 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.434735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.434812 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.434906 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:41.435944 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.437166 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.439819 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.441113 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.441314 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.441398 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.441774 (+ 376us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.441892 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 06:50:41.442895 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.431986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:43297 (local address 127.22.11.130:55921)
0810 06:50:41.434364 (+ 2378us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:41.434405 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:41.434569 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:41.434918 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:41.434921 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:41.434941 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:41.435756 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.435771 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.437337 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.437341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.439685 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:41.439697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.441407 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.441418 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:41.441595 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.441619 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:50:41.442370 (+ 751us) client_negotiation.cc:240] Negotiation successful
0810 06:50:41.442493 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2215,"thread_start_us":1033,"threads_started":1}
I20260810 06:50:41.443110 2280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0e238a82fc4464977296f738fe59d7" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7da9d94e1ade47dabb9975696e23b7c1" is_pre_election: true
I20260810 06:50:41.443773 2280 raft_consensus.cc:2454] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 0.
I20260810 06:50:41.444079 2781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0e238a82fc4464977296f738fe59d7" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36b16974d96a42b7bf934f165eef02c2" is_pre_election: true
I20260810 06:50:41.444669 2415 leader_election.cc:302] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7da9d94e1ade47dabb9975696e23b7c1, b74fa238fb9948a580de569f784c290f; no voters:
I20260810 06:50:41.444967 2781 raft_consensus.cc:2454] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 0.
I20260810 06:50:41.445256 2927 raft_consensus.cc:2781] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:41.445518 2927 raft_consensus.cc:482] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:41.445701 2927 raft_consensus.cc:3037] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:41.448992 2927 raft_consensus.cc:504] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:41.449649 2927 leader_election.cc:288] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 election: Requested vote from peers 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297), 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:41.450076 2781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0e238a82fc4464977296f738fe59d7" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36b16974d96a42b7bf934f165eef02c2"
I20260810 06:50:41.450147 2280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0e238a82fc4464977296f738fe59d7" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7da9d94e1ade47dabb9975696e23b7c1"
I20260810 06:50:41.450271 2781 raft_consensus.cc:3037] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:41.450372 2280 raft_consensus.cc:3037] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:41.453685 2280 raft_consensus.cc:2454] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 1.
I20260810 06:50:41.453809 2781 raft_consensus.cc:2454] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 1.
I20260810 06:50:41.454093 2415 leader_election.cc:302] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7da9d94e1ade47dabb9975696e23b7c1, b74fa238fb9948a580de569f784c290f; no voters:
I20260810 06:50:41.454488 2927 raft_consensus.cc:2781] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:41.455418 2927 raft_consensus.cc:686] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 1 LEADER]: Becoming Leader. State: Replica: b74fa238fb9948a580de569f784c290f, State: Running, Role: LEADER
I20260810 06:50:41.456246 2927 consensus_queue.cc:237] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
W20260810 06:50:41.459833 2616 tablet.cc:2369] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:50:41.462787 671 catalog_manager.cc:5495] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f reported cstate change: term changed from 0 to 1, leader changed from <none> to b74fa238fb9948a580de569f784c290f (127.22.11.130). New cstate: current_term: 1 leader_uuid: "b74fa238fb9948a580de569f784c290f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } health_report { overall_health: UNKNOWN } } }
I20260810 06:50:41.497236 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.489741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57868 (local address 127.22.11.130:33987)
0810 06:50:41.489923 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.489927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.490337 (+ 410us) server_negotiation.cc:407] Connection header received
0810 06:50:41.490518 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.490522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.490596 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.490704 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:41.491279 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.492259 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.494708 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.495244 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.495337 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.496018 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:41.496495 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:41.496600 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.496756 (+ 156us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.496838 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:50:41.508019 2781 raft_consensus.cc:1261] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Refusing update from remote peer b74fa238fb9948a580de569f784c290f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:41.508019 2280 raft_consensus.cc:1261] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Refusing update from remote peer b74fa238fb9948a580de569f784c290f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:41.509275 2927 consensus_queue.cc:1035] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:41.509663 2933 consensus_queue.cc:1035] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [LEADER]: Connected to new peer: Peer: permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:41.526109 2941 mvcc.cc:205] Tried to move back new op lower bound from 7316867651605090304 to 7316867651409596416. Current Snapshot: MvccSnapshot[applied={T|T < 7316867651605090304}]
I20260810 06:50:41.550913 674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:41.551335 674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:41.558601 674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1d212eb435b4fe99b8df6f621429257: Sending DeleteTablet for 3 replicas of tablet bd0e238a82fc4464977296f738fe59d7
I20260810 06:50:41.559886 2746 tablet_service.cc:1514] Processing DeleteTablet for tablet bd0e238a82fc4464977296f738fe59d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:41 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:40566
I20260810 06:50:41.561548 2498 tablet_service.cc:1514] Processing DeleteTablet for tablet bd0e238a82fc4464977296f738fe59d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:41 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:57862
I20260810 06:50:41.561596 2254 tablet_service.cc:1514] Processing DeleteTablet for tablet bd0e238a82fc4464977296f738fe59d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:41 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:38606
I20260810 06:50:41.573941 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 620
I20260810 06:50:41.575182 731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:41.577673 2960 tablet_replica.cc:330] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: stopping tablet replica
I20260810 06:50:41.578240 2960 raft_consensus.cc:2229] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:41.579218 2960 raft_consensus.cc:2258] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:41.580329 2961 tablet_replica.cc:330] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: stopping tablet replica
I20260810 06:50:41.580888 2961 raft_consensus.cc:2229] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:41.581735 2960 ts_tablet_manager.cc:1905] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:41.581861 2961 raft_consensus.cc:2258] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:41.583033 2956 tablet_replica.cc:330] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: stopping tablet replica
I20260810 06:50:41.583501 2956 raft_consensus.cc:2229] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:41.584223 2956 raft_consensus.cc:2258] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:41.584450 2961 ts_tablet_manager.cc:1905] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:41.586266 2956 ts_tablet_manager.cc:1905] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:41.595351 2956 ts_tablet_manager.cc:1918] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:41.595602 2956 log.cc:1196] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/bd0e238a82fc4464977296f738fe59d7
I20260810 06:50:41.596501 2956 ts_tablet_manager.cc:1939] T bd0e238a82fc4464977296f738fe59d7 P 36b16974d96a42b7bf934f165eef02c2: Deleting consensus metadata
I20260810 06:50:41.601140 2961 ts_tablet_manager.cc:1918] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:41.601372 2961 log.cc:1196] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/bd0e238a82fc4464977296f738fe59d7
I20260810 06:50:41.602365 2961 ts_tablet_manager.cc:1939] T bd0e238a82fc4464977296f738fe59d7 P b74fa238fb9948a580de569f784c290f: Deleting consensus metadata
I20260810 06:50:41.604162 2960 ts_tablet_manager.cc:1918] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:41.604418 2960 log.cc:1196] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/bd0e238a82fc4464977296f738fe59d7
I20260810 06:50:41.605343 2960 ts_tablet_manager.cc:1939] T bd0e238a82fc4464977296f738fe59d7 P 7da9d94e1ade47dabb9975696e23b7c1: Deleting consensus metadata
I20260810 06:50:41.606359 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.565065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.22.11.189:41675)
0810 06:50:41.565235 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.565241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.573993 (+ 8752us) server_negotiation.cc:407] Connection header received
0810 06:50:41.575481 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.575487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.575600 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.575708 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.580496 (+ 4788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.581579 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.586119 (+ 4540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.586747 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.586868 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.596573 (+ 9705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.596582 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.596587 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.596654 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.601556 (+ 4902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.602717 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.602723 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.602726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.602935 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.603555 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.603561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.603565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.604068 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.604547 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.605499 (+ 952us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.605848 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:50:41.626134 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.572665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.22.11.188:42527)
0810 06:50:41.572837 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.572844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.609326 (+ 36482us) server_negotiation.cc:407] Connection header received
0810 06:50:41.609461 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.609467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.609591 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.609684 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.611943 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.613130 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.614970 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.615428 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.615556 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.618849 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.618856 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.618880 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.618923 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.623085 (+ 4162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.623779 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.623785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.623789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.623958 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.624369 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.624374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.624376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.624875 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.625120 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.625507 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.625747 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:50:41.765659 645 catalog_manager.cc:4841] TS 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747): tablet bd0e238a82fc4464977296f738fe59d7 (table test-table [id=63120499b9fc4cfcbc631caf9e431998]) successfully deleted
I20260810 06:50:41.765812 647 catalog_manager.cc:4841] TS b74fa238fb9948a580de569f784c290f (127.22.11.130:33987): tablet bd0e238a82fc4464977296f738fe59d7 (table test-table [id=63120499b9fc4cfcbc631caf9e431998]) successfully deleted
I20260810 06:50:41.766155 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 620
I20260810 06:50:41.818131 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.798393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.22.11.189:41675)
0810 06:50:41.798610 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.798616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.800205 (+ 1589us) server_negotiation.cc:407] Connection header received
0810 06:50:41.800515 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.800521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.800609 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.800738 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.803657 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.804781 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.806813 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.807512 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.807608 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.811144 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.811151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.811157 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.811199 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.814473 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.815532 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.815538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.815542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.815690 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.816378 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.816383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.816387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.816755 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.816997 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.817561 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.817785 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:50:41.820863 2862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:41.800484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44940 (local address 127.22.11.188:42527)
0810 06:50:41.800711 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:41.800717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:41.803988 (+ 3271us) server_negotiation.cc:407] Connection header received
0810 06:50:41.804238 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:41.804244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:41.804322 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:41.804406 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:41.807011 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.808078 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.810002 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:41.810688 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:41.810789 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:41.814080 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:41.814087 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:41.814091 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:41.814132 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:41.817489 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.818392 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.818397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.818400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.818542 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:41.818987 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:41.818993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:41.818996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:41.819368 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:41.819607 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:41.820205 (+ 598us) server_negotiation.cc:299] Negotiation successful
0810 06:50:41.820473 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:50:42.533241 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:42.532471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:35765)
0810 06:50:42.532934 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:42.533074 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":246,"threads_started":1}
W20260810 06:50:42.533699 2614 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:38069 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:42.541441 2975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:42.540743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:53687)
0810 06:50:42.541174 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:42.541289 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":237,"threads_started":1}
W20260810 06:50:42.541852 2356 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:38069 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:42.543133 2976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:42.542312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:60837)
0810 06:50:42.542778 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:42.542919 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":241,"threads_started":1}
W20260810 06:50:42.543443 2859 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:38069 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:43.059805 2977 raft_consensus.cc:482] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f1d212eb435b4fe99b8df6f621429257)
I20260810 06:50:43.060106 2977 raft_consensus.cc:504] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:43.060981 2978 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f1d212eb435b4fe99b8df6f621429257)
I20260810 06:50:43.061203 2978 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:43.061794 2977 leader_election.cc:288] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069), bb98122a9ca4498bb89cd5c70007e702 (127.22.11.189:41675)
I20260810 06:50:43.062976 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.061865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32878)
0810 06:50:43.062525 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.062719 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":395,"threads_started":1}
W20260810 06:50:43.063352 2013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:43.063479 2978 leader_election.cc:288] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069), f14f57b0e85e4c49898723fda3c31986 (127.22.11.188:42527)
I20260810 06:50:43.063604 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.062044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32882)
0810 06:50:43.063137 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.063316 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":990,"thread_start_us":858,"threads_started":1}
W20260810 06:50:43.064719 1692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:43.065466 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.064913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32898)
0810 06:50:43.065223 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.065314 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
W20260810 06:50:43.065685 2013 leader_election.cc:334] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069): Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:43.069100 2991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.068068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32912)
0810 06:50:43.068648 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.068828 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":249,"threads_started":1}
W20260810 06:50:43.069339 1692 leader_election.cc:334] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069): Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:43.071271 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.062043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:41675 (local address 127.0.0.1:51328)
0810 06:50:43.062829 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.062846 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:43.062982 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:43.065660 (+ 2678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:43.065669 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:43.065698 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:43.066116 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:43.066129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.067334 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.067339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:43.069139 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:43.069155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.070422 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.070429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:43.070640 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:43.070687 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 06:50:43.070767 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:50:43.070887 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":3,"thread_start_us":531,"threads_started":1}
I20260810 06:50:43.071298 2984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.063150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.22.11.189:41675)
0810 06:50:43.064880 (+ 1730us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:43.064886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:43.064991 (+ 105us) server_negotiation.cc:407] Connection header received
0810 06:50:43.065070 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:43.065075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:43.065185 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:43.065289 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:43.066341 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.067181 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.069329 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.070239 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.070352 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:43.070449 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:43.070810 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 06:50:43.070949 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1594,"thread_start_us":1284,"threads_started":1}
I20260810 06:50:43.071583 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.063998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42527 (local address 127.0.0.1:44944)
0810 06:50:43.064197 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.064216 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:43.064339 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:43.065617 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:43.065623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:43.065652 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:43.065957 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:43.065969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.067756 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.067761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:43.069627 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:43.069639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.070887 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.070894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:43.071078 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:43.071105 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 06:50:43.071198 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 06:50:43.071337 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 06:50:43.072041 2988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.064381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44944 (local address 127.22.11.188:42527)
0810 06:50:43.065220 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:43.065226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:43.065251 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:50:43.065316 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:43.065322 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:43.065409 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:43.066105 (+ 696us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:43.066311 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.067620 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.069774 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:43.070709 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:43.070818 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:43.070938 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:43.071253 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 06:50:43.071628 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":696,"thread_start_us":301,"threads_started":1}
I20260810 06:50:43.072080 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f14f57b0e85e4c49898723fda3c31986" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb98122a9ca4498bb89cd5c70007e702" is_pre_election: true
I20260810 06:50:43.072386 1735 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f14f57b0e85e4c49898723fda3c31986 in term 1.
I20260810 06:50:43.072577 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb98122a9ca4498bb89cd5c70007e702" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f14f57b0e85e4c49898723fda3c31986" is_pre_election: true
I20260810 06:50:43.072921 2058 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb98122a9ca4498bb89cd5c70007e702 in term 1.
I20260810 06:50:43.072893 2016 leader_election.cc:302] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb98122a9ca4498bb89cd5c70007e702, f14f57b0e85e4c49898723fda3c31986; no voters: f1d212eb435b4fe99b8df6f621429257
I20260810 06:50:43.073298 2977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:50:43.073436 2977 raft_consensus.cc:482] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f1d212eb435b4fe99b8df6f621429257)
I20260810 06:50:43.073415 1696 leader_election.cc:302] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb98122a9ca4498bb89cd5c70007e702, f14f57b0e85e4c49898723fda3c31986; no voters: f1d212eb435b4fe99b8df6f621429257
I20260810 06:50:43.073539 2977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:50:43.073702 2978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 06:50:43.073814 2978 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f1d212eb435b4fe99b8df6f621429257)
I20260810 06:50:43.073905 2978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:50:43.075642 2977 raft_consensus.cc:504] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:43.075904 2978 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:43.076238 2977 leader_election.cc:288] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [CANDIDATE]: Term 2 election: Requested vote from peers f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069), bb98122a9ca4498bb89cd5c70007e702 (127.22.11.189:41675)
I20260810 06:50:43.076519 2978 leader_election.cc:288] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 2 election: Requested vote from peers f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069), f14f57b0e85e4c49898723fda3c31986 (127.22.11.188:42527)
I20260810 06:50:43.076920 1735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f14f57b0e85e4c49898723fda3c31986" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bb98122a9ca4498bb89cd5c70007e702"
I20260810 06:50:43.076931 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb98122a9ca4498bb89cd5c70007e702" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f14f57b0e85e4c49898723fda3c31986"
I20260810 06:50:43.077075 2980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.076367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32922)
0810 06:50:43.076795 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.076896 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260810 06:50:43.077176 1735 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f14f57b0e85e4c49898723fda3c31986 in current term 2: Already voted for candidate bb98122a9ca4498bb89cd5c70007e702 in this term.
I20260810 06:50:43.077252 2058 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bb98122a9ca4498bb89cd5c70007e702 in current term 2: Already voted for candidate f14f57b0e85e4c49898723fda3c31986 in this term.
I20260810 06:50:43.077813 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.076569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32934)
0810 06:50:43.077578 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.077653 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":918}
I20260810 06:50:43.077994 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.077629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32946)
0810 06:50:43.077756 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.077837 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 06:50:43.078142 2013 leader_election.cc:334] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069): Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:43.078248 2013 leader_election.cc:302] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f14f57b0e85e4c49898723fda3c31986; no voters: bb98122a9ca4498bb89cd5c70007e702, f1d212eb435b4fe99b8df6f621429257
I20260810 06:50:43.078495 2977 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 06:50:43.078897 2983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.078587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:32948)
0810 06:50:43.078720 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.078786 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:50:43.079046 1692 leader_election.cc:334] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069): Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:43.079145 1692 leader_election.cc:302] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bb98122a9ca4498bb89cd5c70007e702; no voters: f14f57b0e85e4c49898723fda3c31986, f1d212eb435b4fe99b8df6f621429257
I20260810 06:50:43.079368 2978 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260810 06:50:43.535135 2992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.534416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:46893)
0810 06:50:43.534852 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.534935 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":230,"threads_started":1}
I20260810 06:50:43.543084 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.542399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:60571)
0810 06:50:43.542794 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.542906 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":209,"threads_started":1}
I20260810 06:50:43.544837 2994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:43.544006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:58005)
0810 06:50:43.544492 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:43.544642 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":250,"threads_started":1}
I20260810 06:50:44.536741 2996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.535989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:36375)
0810 06:50:44.536428 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.536523 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":235,"threads_started":1}
I20260810 06:50:44.544483 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.543878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:42557)
0810 06:50:44.544248 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.544335 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":192,"threads_started":1}
I20260810 06:50:44.546340 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.545636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:56153)
0810 06:50:44.546120 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.546195 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":293,"threads_started":1}
I20260810 06:50:44.606362 3000 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:44.606559 3000 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:44.607161 3000 leader_election.cc:288] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069), f14f57b0e85e4c49898723fda3c31986 (127.22.11.188:42527)
I20260810 06:50:44.607687 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb98122a9ca4498bb89cd5c70007e702" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f14f57b0e85e4c49898723fda3c31986" is_pre_election: true
I20260810 06:50:44.607959 2058 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb98122a9ca4498bb89cd5c70007e702 in term 2.
I20260810 06:50:44.608139 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.607280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33140)
0810 06:50:44.607754 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.607916 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":234,"threads_started":1}
I20260810 06:50:44.608425 1696 leader_election.cc:302] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb98122a9ca4498bb89cd5c70007e702, f14f57b0e85e4c49898723fda3c31986; no voters:
I20260810 06:50:44.608767 3000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260810 06:50:44.608908 3000 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:44.609018 3000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:50:44.609568 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.609195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33156)
0810 06:50:44.609354 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.609440 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:50:44.609745 1692 leader_election.cc:334] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069): Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:44.611181 3000 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:44.611672 3000 leader_election.cc:288] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 3 election: Requested vote from peers f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069), f14f57b0e85e4c49898723fda3c31986 (127.22.11.188:42527)
I20260810 06:50:44.612097 2058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bb98122a9ca4498bb89cd5c70007e702" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f14f57b0e85e4c49898723fda3c31986"
I20260810 06:50:44.612195 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.611822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33158)
0810 06:50:44.611995 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.612085 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:50:44.612286 2058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:50:44.613081 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.612758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33170)
0810 06:50:44.612906 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.612987 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260810 06:50:44.613245 1692 leader_election.cc:334] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069): Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
I20260810 06:50:44.614465 2058 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb98122a9ca4498bb89cd5c70007e702 in term 3.
I20260810 06:50:44.614804 1696 leader_election.cc:302] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bb98122a9ca4498bb89cd5c70007e702, f14f57b0e85e4c49898723fda3c31986; no voters: f1d212eb435b4fe99b8df6f621429257
I20260810 06:50:44.615065 3000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:50:44.615464 3000 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [term 3 LEADER]: Becoming Leader. State: Replica: bb98122a9ca4498bb89cd5c70007e702, State: Running, Role: LEADER
I20260810 06:50:44.615900 3000 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } }
I20260810 06:50:44.618010 3003 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb98122a9ca4498bb89cd5c70007e702. Latest consensus state: current_term: 3 leader_uuid: "bb98122a9ca4498bb89cd5c70007e702" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:44.618594 3003 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:44.619268 3006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:44.623332 3006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:44.624199 3006 catalog_manager.cc:1259] Loaded cluster ID: 4d8b3452f15a4a3ea68023252ba6e2d8
I20260810 06:50:44.624267 3006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:44.625452 3006 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:44.626250 3006 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Loaded TSK: 0
I20260810 06:50:44.627192 3006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:50:44.680814 1716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:44.687726 1716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:44.691769 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.691336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33196)
0810 06:50:44.691516 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.691610 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 06:50:44.691952 2058 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [term 3 FOLLOWER]: Refusing update from remote peer bb98122a9ca4498bb89cd5c70007e702: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260810 06:50:44.692585 3003 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260810 06:50:44.692766 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.692370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33212)
0810 06:50:44.692536 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.692605 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 06:50:44.692951 1692 consensus_peers.cc:487] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 -> Peer f1d212eb435b4fe99b8df6f621429257 (127.22.11.190:38069): Couldn't send request to peer f1d212eb435b4fe99b8df6f621429257. Status: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 06:50:44.695797 3009 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb98122a9ca4498bb89cd5c70007e702. Latest consensus state: current_term: 3 leader_uuid: "bb98122a9ca4498bb89cd5c70007e702" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:44.696020 3009 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:50:44.696830 3003 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "bb98122a9ca4498bb89cd5c70007e702" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:44.697089 3003 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:44.697398 3009 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "bb98122a9ca4498bb89cd5c70007e702" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "f1d212eb435b4fe99b8df6f621429257" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 38069 } } peers { permanent_uuid: "bb98122a9ca4498bb89cd5c70007e702" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 41675 } attrs { promote: false } } peers { permanent_uuid: "f14f57b0e85e4c49898723fda3c31986" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42527 } attrs { promote: false } } }
I20260810 06:50:44.697741 3009 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14f57b0e85e4c49898723fda3c31986 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:50:44.732882 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.705111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:33987 (local address 127.0.0.1:57882)
0810 06:50:44.705398 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.705418 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:44.705529 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:44.727269 (+ 21740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:44.727274 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:44.727303 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:44.727693 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:44.727705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.729008 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.729013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:44.730560 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:44.730572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.732005 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.732010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:44.732200 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.732273 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 06:50:44.732372 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 06:50:44.732502 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":50}
I20260810 06:50:44.732928 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.705400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.22.11.130:33987)
0810 06:50:44.726948 (+ 21548us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:44.726953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:44.726968 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:50:44.727025 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:44.727028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:44.727098 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:44.727837 (+ 739us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:44.728073 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.728882 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.730714 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.731833 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.731951 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.732048 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:44.732415 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 06:50:44.732563 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21439,"thread_start_us":21284,"threads_started":1}
I20260810 06:50:44.733342 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.705840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:43297 (local address 127.0.0.1:40580)
0810 06:50:44.725597 (+ 19757us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.725623 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:44.725781 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:44.726319 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:44.726323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:44.726344 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:44.726714 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:44.726727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.728198 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.728204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:44.730602 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:44.730618 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.732210 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.732216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:44.732417 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.732447 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:50:44.732519 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:50:44.732656 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19630,"mutex_wait_us":119,"thread_start_us":20185,"threads_started":1}
I20260810 06:50:44.733603 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.706220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.22.11.129:33747)
0810 06:50:44.716263 (+ 10043us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:44.716270 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:44.726385 (+ 10115us) server_negotiation.cc:407] Connection header received
0810 06:50:44.726535 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:44.726540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:44.726626 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:44.726725 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:44.727539 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.728557 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.731234 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.732391 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.732501 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.732599 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:44.733165 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 06:50:44.733294 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9897,"thread_start_us":9777,"threads_started":1}
I20260810 06:50:44.733680 2498 tablet_service.cc:1467] Processing CreateTablet for tablet 10ffc4136f9c4cc492b77d40beffe39c (DEFAULT_TABLE table=test-table [id=9825a2ba0d2542c8bedca489c9ff2962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:44.733992 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.706322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.22.11.131:43297)
0810 06:50:44.725132 (+ 18810us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:44.725139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:44.725729 (+ 590us) server_negotiation.cc:407] Connection header received
0810 06:50:44.725927 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:44.725933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:44.726046 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:44.726146 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:44.726963 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.728056 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.730790 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.732007 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.732125 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.732231 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:44.733533 (+ 1302us) server_negotiation.cc:299] Negotiation successful
0810 06:50:44.733672 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18675,"thread_start_us":18474,"threads_started":1}
I20260810 06:50:44.734172 2498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ffc4136f9c4cc492b77d40beffe39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:44.734304 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.705896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:33747 (local address 127.0.0.1:38612)
0810 06:50:44.726271 (+ 20375us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.726289 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:44.726408 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:44.726937 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:44.726941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:44.726965 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:44.727337 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:44.727350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.728758 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.728763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:44.731061 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:44.731074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.732788 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.732795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:44.732971 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.733012 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 06:50:44.733831 (+ 819us) client_negotiation.cc:240] Negotiation successful
0810 06:50:44.733953 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20249,"mutex_wait_us":28,"thread_start_us":19432,"threads_started":1}
I20260810 06:50:44.734760 2746 tablet_service.cc:1467] Processing CreateTablet for tablet 10ffc4136f9c4cc492b77d40beffe39c (DEFAULT_TABLE table=test-table [id=9825a2ba0d2542c8bedca489c9ff2962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:44.735209 2746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ffc4136f9c4cc492b77d40beffe39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:44.735647 2254 tablet_service.cc:1467] Processing CreateTablet for tablet 10ffc4136f9c4cc492b77d40beffe39c (DEFAULT_TABLE table=test-table [id=9825a2ba0d2542c8bedca489c9ff2962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:44.736063 2254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10ffc4136f9c4cc492b77d40beffe39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:44.741110 3027 tablet_bootstrap.cc:492] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: Bootstrap starting.
I20260810 06:50:44.742057 3026 tablet_bootstrap.cc:492] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: Bootstrap starting.
I20260810 06:50:44.742852 3027 tablet_bootstrap.cc:654] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:44.742769 3029 tablet_bootstrap.cc:492] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: Bootstrap starting.
I20260810 06:50:44.743870 3026 tablet_bootstrap.cc:654] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:44.744344 3029 tablet_bootstrap.cc:654] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:44.745051 3027 tablet_bootstrap.cc:492] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: No bootstrap required, opened a new log
I20260810 06:50:44.745189 3027 ts_tablet_manager.cc:1397] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:50:44.745857 3026 tablet_bootstrap.cc:492] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: No bootstrap required, opened a new log
I20260810 06:50:44.746009 3026 ts_tablet_manager.cc:1397] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 06:50:44.746624 3029 tablet_bootstrap.cc:492] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: No bootstrap required, opened a new log
I20260810 06:50:44.746541 3027 raft_consensus.cc:348] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.746758 3027 raft_consensus.cc:374] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:44.746765 3029 ts_tablet_manager.cc:1397] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260810 06:50:44.746865 3027 raft_consensus.cc:729] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7da9d94e1ade47dabb9975696e23b7c1, State: Initialized, Role: FOLLOWER
I20260810 06:50:44.746801 3026 raft_consensus.cc:348] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.747079 3026 raft_consensus.cc:374] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:44.747174 3026 raft_consensus.cc:729] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74fa238fb9948a580de569f784c290f, State: Initialized, Role: FOLLOWER
I20260810 06:50:44.747102 3027 consensus_queue.cc:260] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.747423 3026 consensus_queue.cc:260] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.747522 3029 raft_consensus.cc:348] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.747761 3029 raft_consensus.cc:374] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:44.747869 3029 raft_consensus.cc:729] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b16974d96a42b7bf934f165eef02c2, State: Initialized, Role: FOLLOWER
I20260810 06:50:44.747939 3027 ts_tablet_manager.cc:1428] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:44.748111 3029 consensus_queue.cc:260] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.748919 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.748453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:59491)
0810 06:50:44.748700 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.748782 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260810 06:50:44.752010 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.751640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:57205)
0810 06:50:44.751828 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.751901 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 06:50:44.752975 3026 ts_tablet_manager.cc:1428] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260810 06:50:44.753134 2360 heartbeater.cc:503] Master 127.22.11.189:41675 was elected leader, sending a full tablet report...
I20260810 06:50:44.753222 3029 ts_tablet_manager.cc:1428] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260810 06:50:44.754551 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.754105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:52175)
0810 06:50:44.754315 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.754400 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260810 06:50:44.755849 2996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.755411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:60335)
0810 06:50:44.755608 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.755701 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:50:44.757442 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.756905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:47927)
0810 06:50:44.757216 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.757296 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
I20260810 06:50:44.758006 2996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.757486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:57563)
0810 06:50:44.757801 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.757881 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260810 06:50:44.758279 2860 heartbeater.cc:503] Master 127.22.11.189:41675 was elected leader, sending a full tablet report...
I20260810 06:50:44.841329 3039 raft_consensus.cc:482] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:44.841560 3039 raft_consensus.cc:504] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.842269 3039 leader_election.cc:288] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297), 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:44.842831 2781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ffc4136f9c4cc492b77d40beffe39c" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36b16974d96a42b7bf934f165eef02c2" is_pre_election: true
I20260810 06:50:44.842856 2280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ffc4136f9c4cc492b77d40beffe39c" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7da9d94e1ade47dabb9975696e23b7c1" is_pre_election: true
I20260810 06:50:44.843111 2280 raft_consensus.cc:2454] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 0.
I20260810 06:50:44.843111 2781 raft_consensus.cc:2454] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 0.
I20260810 06:50:44.843617 2415 leader_election.cc:302] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7da9d94e1ade47dabb9975696e23b7c1, b74fa238fb9948a580de569f784c290f; no voters:
I20260810 06:50:44.843883 3039 raft_consensus.cc:2781] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:44.843994 3039 raft_consensus.cc:482] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:44.844094 3039 raft_consensus.cc:3037] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:44.846179 3039 raft_consensus.cc:504] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.846691 3039 leader_election.cc:288] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 election: Requested vote from peers 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297), 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747)
I20260810 06:50:44.847179 2781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ffc4136f9c4cc492b77d40beffe39c" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36b16974d96a42b7bf934f165eef02c2"
I20260810 06:50:44.847179 2280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10ffc4136f9c4cc492b77d40beffe39c" candidate_uuid: "b74fa238fb9948a580de569f784c290f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7da9d94e1ade47dabb9975696e23b7c1"
I20260810 06:50:44.847384 2781 raft_consensus.cc:3037] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:44.847384 2280 raft_consensus.cc:3037] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:44.850247 2280 raft_consensus.cc:2454] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 1.
I20260810 06:50:44.850247 2781 raft_consensus.cc:2454] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b74fa238fb9948a580de569f784c290f in term 1.
I20260810 06:50:44.850673 2416 leader_election.cc:302] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 36b16974d96a42b7bf934f165eef02c2, 7da9d94e1ade47dabb9975696e23b7c1, b74fa238fb9948a580de569f784c290f; no voters:
I20260810 06:50:44.851020 3039 raft_consensus.cc:2781] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:44.851435 3039 raft_consensus.cc:686] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 1 LEADER]: Becoming Leader. State: Replica: b74fa238fb9948a580de569f784c290f, State: Running, Role: LEADER
I20260810 06:50:44.851747 2615 heartbeater.cc:503] Master 127.22.11.189:41675 was elected leader, sending a full tablet report...
I20260810 06:50:44.851733 3039 consensus_queue.cc:237] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } }
I20260810 06:50:44.852453 2996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.852023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:46571)
0810 06:50:44.852182 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.852297 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:50:44.854730 1716 catalog_manager.cc:5495] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f reported cstate change: term changed from 0 to 1, leader changed from <none> to b74fa238fb9948a580de569f784c290f (127.22.11.130). New cstate: current_term: 1 leader_uuid: "b74fa238fb9948a580de569f784c290f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74fa238fb9948a580de569f784c290f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 33987 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 } health_report { overall_health: UNKNOWN } } }
I20260810 06:50:44.883008 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.878534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57892 (local address 127.22.11.130:33987)
0810 06:50:44.878727 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:44.878733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:44.878751 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:50:44.878928 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:44.878933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:44.879008 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:44.879081 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:44.879441 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.880142 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.881478 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.881890 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.882009 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.882365 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:44.882475 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:44.882549 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:44.882688 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 06:50:44.882766 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:50:44.885772 2280 raft_consensus.cc:1261] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Refusing update from remote peer b74fa238fb9948a580de569f784c290f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:44.885823 2781 raft_consensus.cc:1261] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Refusing update from remote peer b74fa238fb9948a580de569f784c290f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:44.886360 3043 consensus_queue.cc:1035] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [LEADER]: Connected to new peer: Peer: permanent_uuid: "36b16974d96a42b7bf934f165eef02c2" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:44.886667 3039 consensus_queue.cc:1035] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7da9d94e1ade47dabb9975696e23b7c1" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:44.888823 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.888270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:58245)
0810 06:50:44.888525 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.888643 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260810 06:50:44.889160 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.888795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:54613)
0810 06:50:44.888945 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.889029 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 06:50:44.895556 2997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.894940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.129:54559)
0810 06:50:44.895309 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.895397 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244}
I20260810 06:50:44.896238 2996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.895627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:57657)
0810 06:50:44.895965 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.896056 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232}
I20260810 06:50:44.897433 2999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.896866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.131:39605)
0810 06:50:44.897166 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.897245 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196}
I20260810 06:50:44.899039 2996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.898550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.22.11.130:59155)
0810 06:50:44.898792 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:44.898887 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260810 06:50:44.904944 1717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:44.905167 1717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:44.909909 1717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bb98122a9ca4498bb89cd5c70007e702: Sending DeleteTablet for 3 replicas of tablet 10ffc4136f9c4cc492b77d40beffe39c
I20260810 06:50:44.910789 2498 tablet_service.cc:1514] Processing DeleteTablet for tablet 10ffc4136f9c4cc492b77d40beffe39c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:44 UTC) from {username='oryx', principal='oryx/127.22.11.189@KRBTEST.COM'} at 127.0.0.1:57882
I20260810 06:50:44.910836 2746 tablet_service.cc:1514] Processing DeleteTablet for tablet 10ffc4136f9c4cc492b77d40beffe39c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:44 UTC) from {username='oryx', principal='oryx/127.22.11.189@KRBTEST.COM'} at 127.0.0.1:40580
I20260810 06:50:44.911445 3065 tablet_replica.cc:330] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: stopping tablet replica
I20260810 06:50:44.911634 3065 raft_consensus.cc:2229] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:44.911856 2254 tablet_service.cc:1514] Processing DeleteTablet for tablet 10ffc4136f9c4cc492b77d40beffe39c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:44 UTC) from {username='oryx', principal='oryx/127.22.11.189@KRBTEST.COM'} at 127.0.0.1:38612
I20260810 06:50:44.911918 3065 raft_consensus.cc:2258] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:44.913290 3065 ts_tablet_manager.cc:1905] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:44.913417 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 1988
I20260810 06:50:44.920197 3064 tablet_replica.cc:330] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: stopping tablet replica
I20260810 06:50:44.920384 3064 raft_consensus.cc:2229] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:44.920463 3067 tablet_replica.cc:330] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: stopping tablet replica
I20260810 06:50:44.920675 3067 raft_consensus.cc:2229] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:44.920825 3064 raft_consensus.cc:2258] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:44.921021 3067 raft_consensus.cc:2258] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:44.921523 3065 ts_tablet_manager.cc:1918] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:44.921662 3065 log.cc:1196] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/10ffc4136f9c4cc492b77d40beffe39c
I20260810 06:50:44.922106 3065 ts_tablet_manager.cc:1939] T 10ffc4136f9c4cc492b77d40beffe39c P 36b16974d96a42b7bf934f165eef02c2: Deleting consensus metadata
I20260810 06:50:44.922348 3067 ts_tablet_manager.cc:1905] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:44.923218 1694 catalog_manager.cc:4841] TS 36b16974d96a42b7bf934f165eef02c2 (127.22.11.131:43297): tablet 10ffc4136f9c4cc492b77d40beffe39c (table test-table [id=9825a2ba0d2542c8bedca489c9ff2962]) successfully deleted
I20260810 06:50:44.923687 3064 ts_tablet_manager.cc:1905] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:44.928115 3067 ts_tablet_manager.cc:1918] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:44.928241 3067 log.cc:1196] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/10ffc4136f9c4cc492b77d40beffe39c
I20260810 06:50:44.928606 3067 ts_tablet_manager.cc:1939] T 10ffc4136f9c4cc492b77d40beffe39c P 7da9d94e1ade47dabb9975696e23b7c1: Deleting consensus metadata
I20260810 06:50:44.928758 3064 ts_tablet_manager.cc:1918] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:44.928879 3064 log.cc:1196] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/10ffc4136f9c4cc492b77d40beffe39c
I20260810 06:50:44.929344 3064 ts_tablet_manager.cc:1939] T 10ffc4136f9c4cc492b77d40beffe39c P b74fa238fb9948a580de569f784c290f: Deleting consensus metadata
I20260810 06:50:44.929574 1693 catalog_manager.cc:4841] TS 7da9d94e1ade47dabb9975696e23b7c1 (127.22.11.129:33747): tablet 10ffc4136f9c4cc492b77d40beffe39c (table test-table [id=9825a2ba0d2542c8bedca489c9ff2962]) successfully deleted
I20260810 06:50:44.930284 1694 catalog_manager.cc:4841] TS b74fa238fb9948a580de569f784c290f (127.22.11.130:33987): tablet 10ffc4136f9c4cc492b77d40beffe39c (table test-table [id=9825a2ba0d2542c8bedca489c9ff2962]) successfully deleted
I20260810 06:50:44.932811 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:44.915793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.22.11.129:33747)
0810 06:50:44.915982 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:44.915988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:44.916466 (+ 478us) server_negotiation.cc:407] Connection header received
0810 06:50:44.916751 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:44.916757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:44.916842 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:44.916912 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:44.918947 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.920129 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.922304 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:44.922982 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:44.923131 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:44.926639 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:44.926657 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:44.926678 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:44.926730 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:44.930616 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:44.931216 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:44.931223 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:44.931225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:44.931361 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:44.931645 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:44.931648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:44.931650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:44.931955 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:44.932173 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:44.932357 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 06:50:44.932517 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:50:44.933398 2350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:45.025046 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 1988
I20260810 06:50:45.051905 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 2380
I20260810 06:50:45.065387 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:45.052648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.22.11.130:33987)
0810 06:50:45.052810 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:45.052815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:45.052846 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:50:45.053021 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:45.053025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:45.053098 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:45.053180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:45.054441 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:45.055148 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:45.056277 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:45.056708 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:45.056799 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:45.059095 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:45.059114 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:45.059137 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:45.059187 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:45.062825 (+ 3638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:45.063451 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:45.063461 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:45.063465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:45.063622 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:45.063897 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:45.063901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:45.063905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:45.064327 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:45.064608 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:45.064806 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 06:50:45.065036 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:50:45.066095 2598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:45.149951 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:45.149546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33222)
0810 06:50:45.149705 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:45.149805 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:50:45.150959 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:45.150624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:38069 (local address 127.0.0.1:33238)
0810 06:50:45.150739 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:45.150821 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:38069: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:50:45.184926 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 2380
I20260810 06:50:45.214795 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 2631
I20260810 06:50:45.227028 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:45.215476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.22.11.131:43297)
0810 06:50:45.215633 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:45.215637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:45.215692 (+ 55us) server_negotiation.cc:407] Connection header received
0810 06:50:45.215851 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:45.215854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:45.215918 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:45.215998 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:45.217323 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:45.218009 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:45.219137 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:45.219563 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:45.219650 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:45.222217 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:45.222234 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:45.222251 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:45.222290 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:45.224897 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:45.225445 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:45.225451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:45.225453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:45.225591 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:45.225876 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:45.225879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:45.225880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:45.226191 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:45.226418 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:45.226602 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 06:50:45.226765 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:50:45.227643 2845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:45.323889 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 2631
I20260810 06:50:45.352432 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 1489
I20260810 06:50:45.353500 1771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:45.483168 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 1489
W20260810 06:50:45.511960 2013 connection.cc:535] server connection from 127.0.0.1:44944 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:50:45.512307 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 1654
I20260810 06:50:45.513355 2097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:45.629397 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 1654
2026-08-10T06:50:45Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11407 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:45 dist-test-slave-f7th krb5kdc[3085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:45 dist-test-slave-f7th krb5kdc[3085](info): set up 2 sockets
Aug 10 06:50:45 dist-test-slave-f7th krb5kdc[3085](info): commencing operation
krb5kdc: starting...
W20260810 06:50:46.877974 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:47 dist-test-slave-f7th krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab.
Entry for principal oryx/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab.
I20260810 06:50:47.108086 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:36439
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab
--principal=oryx/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:36439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:47.425343 3105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:47.425626 3105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:47.425755 3105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:47.432776 3105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:47.432874 3105 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 06:50:47.432941 3105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:47.433027 3105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:47.433095 3105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:47.433184 3105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:47.442833 3105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:36439
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.190.keytab
--principal=oryx/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:36439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:47.444034 3105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:47.445330 3105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:47.456257 3114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:47.456308 3115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:47.456569 3117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:47.457157 3105 server_base.cc:1034] running on GCE node
I20260810 06:50:47.457857 3105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:47.459125 3105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:47.460335 3105 hybrid_clock.cc:648] HybridClock initialized: now 1786344647460273 us; error 33 us; skew 500 ppm
I20260810 06:50:47.460650 3105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:47 dist-test-slave-f7th krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344647, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:47.463505 3105 init.cc:377] Logged in from keytab as oryx/127.22.11.190@KRBTEST.COM (short username oryx)
I20260810 06:50:47.464831 3105 webserver.cc:460] Webserver started at http://127.22.11.190:36099/ using document root <none> and password file <none>
I20260810 06:50:47.465474 3105 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:47.465579 3105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:47.465881 3105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:47.468776 3105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "0da5badb4a034d11b180f044322910e1"
format_stamp: "Formatted at 2026-08-10 06:50:47 on dist-test-slave-f7th"
server_key: "245ab01b26136fbb4059f98d79887f87"
server_key_iv: "53fe52f8a48b194bbff6de535e0e371f"
server_key_version: "encryptionkey@0"
I20260810 06:50:47.469542 3105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "0da5badb4a034d11b180f044322910e1"
format_stamp: "Formatted at 2026-08-10 06:50:47 on dist-test-slave-f7th"
server_key: "245ab01b26136fbb4059f98d79887f87"
server_key_iv: "53fe52f8a48b194bbff6de535e0e371f"
server_key_version: "encryptionkey@0"
I20260810 06:50:47.474188 3105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:50:47.477389 3131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:47.478556 3105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:50:47.478714 3105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "0da5badb4a034d11b180f044322910e1"
format_stamp: "Formatted at 2026-08-10 06:50:47 on dist-test-slave-f7th"
server_key: "245ab01b26136fbb4059f98d79887f87"
server_key_iv: "53fe52f8a48b194bbff6de535e0e371f"
server_key_version: "encryptionkey@0"
I20260810 06:50:47.478895 3105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:47.494788 3105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:47.498461 3105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:47.498690 3105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:47.518013 3105 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:36439
I20260810 06:50:47.518023 3247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:36439 every 8 connection(s)
I20260810 06:50:47.519232 3105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:47.522760 3249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:47.527447 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 3105
I20260810 06:50:47.527652 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:47.528280 22574 external_mini_cluster.cc:1428] Setting key 0e709a310c3945916a73d3a753a255ad
I20260810 06:50:47.533063 3249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Bootstrap starting.
I20260810 06:50:47.535955 3249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:47.536937 3249 log.cc:824] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:47.539404 3249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: No bootstrap required, opened a new log
Aug 10 06:50:47 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344647, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:47.546824 3249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da5badb4a034d11b180f044322910e1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 36439 } }
I20260810 06:50:47.547165 3249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:47.547258 3249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da5badb4a034d11b180f044322910e1, State: Initialized, Role: FOLLOWER
I20260810 06:50:47.547879 3249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da5badb4a034d11b180f044322910e1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 36439 } }
I20260810 06:50:47.548121 3249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:47.548259 3249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:47.548437 3249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:47.548996 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:47.530165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.22.11.190:36439)
0810 06:50:47.530932 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:47.530945 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:47.530985 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:50:47.532652 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:47.532677 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:47.533033 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:47.533370 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:47.534750 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:47.536064 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:47.537403 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:47.538070 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:47.538230 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:47.542162 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:47.542169 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:47.542186 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:47.542224 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:47.545162 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:47.545778 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:47.545788 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:47.545794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:47.545957 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:47.546307 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:47.546310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:47.546312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:47.546804 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:47.547097 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:47.547505 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 06:50:47.547874 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":346,"threads_started":1}
I20260810 06:50:47.551112 3249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da5badb4a034d11b180f044322910e1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 36439 } }
I20260810 06:50:47.551630 3249 leader_election.cc:302] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0da5badb4a034d11b180f044322910e1; no voters:
I20260810 06:50:47.552191 3249 leader_election.cc:288] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:47.552299 3262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:47.553354 3262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [term 1 LEADER]: Becoming Leader. State: Replica: 0da5badb4a034d11b180f044322910e1, State: Running, Role: LEADER
I20260810 06:50:47.553958 3262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da5badb4a034d11b180f044322910e1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 36439 } }
I20260810 06:50:47.554194 3249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:47.556659 3263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0da5badb4a034d11b180f044322910e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da5badb4a034d11b180f044322910e1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 36439 } } }
I20260810 06:50:47.557045 3263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:47.556607 3264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0da5badb4a034d11b180f044322910e1. Latest consensus state: current_term: 1 leader_uuid: "0da5badb4a034d11b180f044322910e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da5badb4a034d11b180f044322910e1" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 36439 } } }
I20260810 06:50:47.557379 3264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1 [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:47.558290 3272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:47.562551 3272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:47.570607 3272 catalog_manager.cc:1347] Generated new cluster ID: f0decae1e8d14107a2e3c60b4ddb55e7
I20260810 06:50:47.570698 3272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:47.601280 3272 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:47.602511 3272 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:47.612741 3272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Generated new TSK 0
I20260810 06:50:47.613458 3272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab.
Entry for principal oryx/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab.
I20260810 06:50:47.669669 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:36439
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab
--principal=oryx/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:47.994948 3298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:47.995208 3298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:47.995345 3298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:48.002452 3298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:48.002552 3298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:48.002683 3298 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:50:48.002789 3298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:48.012142 3298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab
--principal=oryx/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:36439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:48.013311 3298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:48.014578 3298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:48.026561 3315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:48.026608 3304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:48.026631 3306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:48.027328 3298 server_base.cc:1034] running on GCE node
I20260810 06:50:48.027902 3298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:48.028748 3298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:48.029946 3298 hybrid_clock.cc:648] HybridClock initialized: now 1786344648029891 us; error 34 us; skew 500 ppm
I20260810 06:50:48.030223 3298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:48 dist-test-slave-f7th krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344648, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:48.033257 3298 init.cc:377] Logged in from keytab as oryx/127.22.11.129@KRBTEST.COM (short username oryx)
I20260810 06:50:48.034583 3298 webserver.cc:460] Webserver started at http://127.22.11.129:46867/ using document root <none> and password file <none>
I20260810 06:50:48.035214 3298 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:48.035315 3298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:48.035619 3298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:48.038410 3298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "f03f5db71d114bf4b2ec97c40ad7d62f"
format_stamp: "Formatted at 2026-08-10 06:50:48 on dist-test-slave-f7th"
server_key: "bb00a228b5e8d95e34b8c49bd3e87984"
server_key_iv: "dc4e1ccacf75b8166df72584c17d23d5"
server_key_version: "encryptionkey@0"
I20260810 06:50:48.039144 3298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "f03f5db71d114bf4b2ec97c40ad7d62f"
format_stamp: "Formatted at 2026-08-10 06:50:48 on dist-test-slave-f7th"
server_key: "bb00a228b5e8d95e34b8c49bd3e87984"
server_key_iv: "dc4e1ccacf75b8166df72584c17d23d5"
server_key_version: "encryptionkey@0"
I20260810 06:50:48.043927 3298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 06:50:48.047159 3328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:48.048499 3298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:50:48.048691 3298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "f03f5db71d114bf4b2ec97c40ad7d62f"
format_stamp: "Formatted at 2026-08-10 06:50:48 on dist-test-slave-f7th"
server_key: "bb00a228b5e8d95e34b8c49bd3e87984"
server_key_iv: "dc4e1ccacf75b8166df72584c17d23d5"
server_key_version: "encryptionkey@0"
I20260810 06:50:48.048892 3298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:48.070302 3298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:48.073838 3298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:48.074059 3298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:48.076575 3298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:48.076704 3298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:50:48.076905 3298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:48.076975 3298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:50:48 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344648, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:48.099911 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.076884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:59205 (local address 127.22.11.190:36439)
0810 06:50:48.079230 (+ 2346us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:48.079234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:48.079252 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:50:48.081354 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:48.081359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:48.081425 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:48.081515 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:48.084086 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.084981 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.087883 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.088354 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.088493 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.093225 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:48.093232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:48.093237 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:48.093282 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:48.096495 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.097458 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.097464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.097469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.097632 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.098066 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.098071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.098075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.098468 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:48.098617 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:48.099233 (+ 616us) server_negotiation.cc:299] Negotiation successful
0810 06:50:48.099497 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2248,"thread_start_us":2158,"threads_started":1}
I20260810 06:50:48.100529 3349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.077545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:36439 (local address 127.22.11.129:59205)
0810 06:50:48.078508 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:48.078567 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:48.081070 (+ 2503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:48.081789 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:48.081804 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:48.082796 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:48.083875 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.083903 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.085194 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.085200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.087711 (+ 2511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.087728 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.088657 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.088666 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.088994 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.089792 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:48.089842 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:48.093001 (+ 3159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:48.096674 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.096684 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.096699 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.097307 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.097742 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.097745 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.097747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.097962 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.098594 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:48.098600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:48.098906 (+ 306us) client_negotiation.cc:769] Sending connection context
0810 06:50:48.099121 (+ 215us) client_negotiation.cc:240] Negotiation successful
0810 06:50:48.099461 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":382,"threads_started":1}
I20260810 06:50:48.119556 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.106389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:44405 (local address 127.22.11.190:36439)
0810 06:50:48.106568 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:48.106574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:48.107111 (+ 537us) server_negotiation.cc:407] Connection header received
0810 06:50:48.107234 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:48.107240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:48.107318 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:48.107430 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:48.108292 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.109499 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.110812 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.111426 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.111553 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.113537 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:48.113543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:48.113549 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:48.113589 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:48.116873 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.117689 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.117695 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.117698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.117858 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.118205 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.118211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.118214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.118458 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:48.118585 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:48.119009 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 06:50:48.119222 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:50:48.119650 3469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.106424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:36439 (local address 127.22.11.129:44405)
0810 06:50:48.107027 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:48.107043 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:48.107144 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:48.107459 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:48.107463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:48.107887 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:48.108175 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.108184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.109657 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.109661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.110674 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.110685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.111628 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.111635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.111792 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.112373 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:48.112397 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:48.113342 (+ 945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:48.117008 (+ 3666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.117013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.117018 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.117524 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.117969 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.117974 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.117978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.118102 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.118577 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:48.118581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:48.118761 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 06:50:48.118893 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 06:50:48.119198 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":307,"threads_started":1}
I20260810 06:50:48.126466 3298 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:36015
I20260810 06:50:48.126487 3535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:36015 every 8 connection(s)
I20260810 06:50:48.127669 3298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:48.129410 3536 heartbeater.cc:347] Connected to a master server at 127.22.11.190:36439
I20260810 06:50:48.129617 3536 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:48.130209 3536 heartbeater.cc:511] Master 127.22.11.190:36439 requested a full tablet report, sending...
I20260810 06:50:48.131592 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 3298
I20260810 06:50:48.131803 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:48.132345 3167 ts_manager.cc:194] Registered new tserver with Master: f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015)
I20260810 06:50:48.132592 22574 external_mini_cluster.cc:1428] Setting key 912a88029fc2f3741e92eeb1f9c253ae
I20260810 06:50:48.134209 3167 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:59205
WARNING: no policy specified for oryx/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab.
Entry for principal oryx/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab.
I20260810 06:50:48.182973 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:36439
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab
--principal=oryx/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:48.502385 3544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:48.502640 3544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:48.502743 3544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:48.509763 3544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:48.509871 3544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:48.509968 3544 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:50:48.510048 3544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:48.519536 3544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab
--principal=oryx/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:36439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:48.520704 3544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:48.521957 3544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:48.534309 3561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:48.534350 3551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:48.534390 3550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:48.535230 3544 server_base.cc:1034] running on GCE node
I20260810 06:50:48.535799 3544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:48.536669 3544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:48.537887 3544 hybrid_clock.cc:648] HybridClock initialized: now 1786344648537822 us; error 40 us; skew 500 ppm
I20260810 06:50:48.538170 3544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:48 dist-test-slave-f7th krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344648, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:48.541232 3544 init.cc:377] Logged in from keytab as oryx/127.22.11.130@KRBTEST.COM (short username oryx)
I20260810 06:50:48.542634 3544 webserver.cc:460] Webserver started at http://127.22.11.130:39905/ using document root <none> and password file <none>
I20260810 06:50:48.543298 3544 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:48.543390 3544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:48.543700 3544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:48.546545 3544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "c1edcf5cefdf49fdaee772112fae5d54"
format_stamp: "Formatted at 2026-08-10 06:50:48 on dist-test-slave-f7th"
server_key: "921e7fdc361e4bfb568144ff9f4c5c53"
server_key_iv: "a39cd6a8cf3c961075ddba546da8b525"
server_key_version: "encryptionkey@0"
I20260810 06:50:48.547318 3544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "c1edcf5cefdf49fdaee772112fae5d54"
format_stamp: "Formatted at 2026-08-10 06:50:48 on dist-test-slave-f7th"
server_key: "921e7fdc361e4bfb568144ff9f4c5c53"
server_key_iv: "a39cd6a8cf3c961075ddba546da8b525"
server_key_version: "encryptionkey@0"
I20260810 06:50:48.552192 3544 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:50:48.555665 3572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:48.556995 3544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:48.557149 3544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "c1edcf5cefdf49fdaee772112fae5d54"
format_stamp: "Formatted at 2026-08-10 06:50:48 on dist-test-slave-f7th"
server_key: "921e7fdc361e4bfb568144ff9f4c5c53"
server_key_iv: "a39cd6a8cf3c961075ddba546da8b525"
server_key_version: "encryptionkey@0"
I20260810 06:50:48.557358 3544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:48.583766 3544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:48.587832 3544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:48.588083 3544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:48.590742 3544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:48.590849 3544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:48.590981 3544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:48.591054 3544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:50:48 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344648, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:48.612524 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.591018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:38081 (local address 127.22.11.190:36439)
0810 06:50:48.591210 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:48.591215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:48.592658 (+ 1443us) server_negotiation.cc:407] Connection header received
0810 06:50:48.594622 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:48.594628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:48.594704 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:48.594793 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:48.596936 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.598098 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.600463 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.600945 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.601079 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.605814 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:48.605821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:48.605826 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:48.605869 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:48.609228 (+ 3359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.610197 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.610202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.610204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.610339 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.610812 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.610815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.610817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.611099 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:48.611220 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:48.612097 (+ 877us) server_negotiation.cc:299] Negotiation successful
0810 06:50:48.612264 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:50:48.614301 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.591694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:36439 (local address 127.22.11.130:38081)
0810 06:50:48.592475 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:48.592517 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:48.594316 (+ 1799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:48.595075 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:48.595087 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:48.595832 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:48.596694 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.596723 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.598279 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.598284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.600307 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.600322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.601196 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.601204 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.601528 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.602339 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:48.602374 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:48.605575 (+ 3201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:48.609374 (+ 3799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.609386 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.609402 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.610033 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.610437 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.610441 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.610445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.610710 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.611221 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:48.611230 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:48.611673 (+ 443us) client_negotiation.cc:769] Sending connection context
0810 06:50:48.611986 (+ 313us) client_negotiation.cc:240] Negotiation successful
0810 06:50:48.612522 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":344,"threads_started":1}
I20260810 06:50:48.638993 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.623546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:34583 (local address 127.22.11.190:36439)
0810 06:50:48.623709 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:48.623714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:48.625070 (+ 1356us) server_negotiation.cc:407] Connection header received
0810 06:50:48.625313 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:48.625318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:48.625387 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:48.625465 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:48.626398 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.627469 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.629704 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.630313 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.630451 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.632997 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:48.633003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:48.633008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:48.633046 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:48.635736 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.636484 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.636489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.636492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.636675 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:48.637220 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:48.637224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:48.637228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:48.637676 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:48.637828 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:48.638445 (+ 617us) server_negotiation.cc:299] Negotiation successful
0810 06:50:48.638682 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:50:48.639243 3719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:48.623669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:36439 (local address 127.22.11.130:34583)
0810 06:50:48.624962 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:48.624983 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:48.625191 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:48.625516 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:48.625521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:48.625974 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:48.626252 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.626260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.627631 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.627640 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.629518 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:48.629536 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:48.630696 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:48.630702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:48.630869 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:48.631569 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:48.631591 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:48.632760 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:48.635857 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.635862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.635866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.636360 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.636813 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:48.636819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:48.636823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:48.637018 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:48.637816 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:48.637826 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:48.638104 (+ 278us) client_negotiation.cc:769] Sending connection context
0810 06:50:48.638280 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 06:50:48.638665 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"thread_start_us":865,"threads_started":1}
I20260810 06:50:48.641505 3544 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:40647
I20260810 06:50:48.641523 3782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:40647 every 8 connection(s)
I20260810 06:50:48.643025 3544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:48.643620 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 3544
I20260810 06:50:48.643800 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:48.644358 22574 external_mini_cluster.cc:1428] Setting key b83455f61c3461d17cab6ed5b5667679
I20260810 06:50:48.645167 3783 heartbeater.cc:347] Connected to a master server at 127.22.11.190:36439
I20260810 06:50:48.645390 3783 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:48.646056 3783 heartbeater.cc:511] Master 127.22.11.190:36439 requested a full tablet report, sending...
I20260810 06:50:48.647291 3167 ts_manager.cc:194] Registered new tserver with Master: c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647)
I20260810 06:50:48.648088 3167 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:38081
WARNING: no policy specified for oryx/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab.
Entry for principal oryx/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab.
I20260810 06:50:48.695655 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:36439
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab
--principal=oryx/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:49.017792 3789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:49.018064 3789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:49.018201 3789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:49.025310 3789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:49.025411 3789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:49.025537 3789 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 06:50:49.025640 3789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:49.035022 3789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab
--principal=oryx/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.22.11.190:36439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:49.036259 3789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:49.037592 3789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:49.049881 3801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:49.049912 3798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:49.050256 3797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:49.050531 3789 server_base.cc:1034] running on GCE node
I20260810 06:50:49.051091 3789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:49.051925 3789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:49.053130 3789 hybrid_clock.cc:648] HybridClock initialized: now 1786344649053061 us; error 39 us; skew 500 ppm
I20260810 06:50:49.053401 3789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:49 dist-test-slave-f7th krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344649, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:49.061005 3789 init.cc:377] Logged in from keytab as oryx/127.22.11.131@KRBTEST.COM (short username oryx)
I20260810 06:50:49.062325 3789 webserver.cc:460] Webserver started at http://127.22.11.131:39129/ using document root <none> and password file <none>
I20260810 06:50:49.062952 3789 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:49.063061 3789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:49.063360 3789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:49.066254 3789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "98789c264f074fdf85c974e311e05fb5"
format_stamp: "Formatted at 2026-08-10 06:50:49 on dist-test-slave-f7th"
server_key: "1b26278213370aacad991ee12a934023"
server_key_iv: "cd2fdd0ab75ed901abadf6b8a383b30c"
server_key_version: "encryptionkey@0"
I20260810 06:50:49.067054 3789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "98789c264f074fdf85c974e311e05fb5"
format_stamp: "Formatted at 2026-08-10 06:50:49 on dist-test-slave-f7th"
server_key: "1b26278213370aacad991ee12a934023"
server_key_iv: "cd2fdd0ab75ed901abadf6b8a383b30c"
server_key_version: "encryptionkey@0"
I20260810 06:50:49.071776 3789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260810 06:50:49.075017 3815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:49.076211 3789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:50:49.076349 3789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "98789c264f074fdf85c974e311e05fb5"
format_stamp: "Formatted at 2026-08-10 06:50:49 on dist-test-slave-f7th"
server_key: "1b26278213370aacad991ee12a934023"
server_key_iv: "cd2fdd0ab75ed901abadf6b8a383b30c"
server_key_version: "encryptionkey@0"
I20260810 06:50:49.076545 3789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:49.095058 3789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:49.098744 3789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:49.098982 3789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:49.101588 3789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:49.101671 3789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:49.101809 3789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:49.101925 3789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 06:50:49 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344649, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:49.124428 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:49.102086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:57099 (local address 127.22.11.190:36439)
0810 06:50:49.102287 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:49.102312 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:49.103462 (+ 1150us) server_negotiation.cc:407] Connection header received
0810 06:50:49.105544 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:49.105550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:49.105629 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:49.105715 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:49.107662 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.108846 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.111101 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.111551 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.111680 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:49.116461 (+ 4781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:49.116467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:49.116472 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:49.116519 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:49.120259 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:49.121344 (+ 1085us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:49.121349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:49.121352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:49.121493 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:49.122111 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:49.122118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:49.122122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:49.122496 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:49.122651 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:49.123784 (+ 1133us) server_negotiation.cc:299] Negotiation successful
0810 06:50:49.124036 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:50:49.128942 3830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:49.102455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:36439 (local address 127.22.11.131:57099)
0810 06:50:49.103286 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:49.103340 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:49.105272 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:49.105955 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:49.105964 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:49.106645 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:49.107413 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:49.107453 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.109040 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.109046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:49.110946 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:49.110962 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.111964 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.111970 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:49.112162 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:49.112932 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:49.112969 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:49.116214 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:49.120444 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:49.120458 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:49.120474 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:49.121211 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:49.121617 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:49.121626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:49.121630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:49.121941 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:49.122737 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:49.122745 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:49.123292 (+ 547us) client_negotiation.cc:769] Sending connection context
0810 06:50:49.123625 (+ 333us) client_negotiation.cc:240] Negotiation successful
0810 06:50:49.124207 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":337,"threads_started":1}
I20260810 06:50:49.136831 3536 heartbeater.cc:503] Master 127.22.11.190:36439 was elected leader, sending a full tablet report...
I20260810 06:50:49.145530 3789 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:39037
I20260810 06:50:49.145567 4026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:39037 every 8 connection(s)
I20260810 06:50:49.147023 3789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:49.147475 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 3789
I20260810 06:50:49.147641 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:49.148239 22574 external_mini_cluster.cc:1428] Setting key 310c0da8391d208687b334cb00b96a09
I20260810 06:50:49.149501 4029 heartbeater.cc:347] Connected to a master server at 127.22.11.190:36439
I20260810 06:50:49.149789 4029 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:49.150715 4029 heartbeater.cc:511] Master 127.22.11.190:36439 requested a full tablet report, sending...
I20260810 06:50:49.152390 3166 ts_manager.cc:194] Registered new tserver with Master: 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037)
I20260810 06:50:49.153457 3166 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:57099
I20260810 06:50:49.155711 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:49.140653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:36439 (local address 127.22.11.131:57949)
0810 06:50:49.141738 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:49.141762 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:49.141915 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:49.142306 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:49.142311 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:49.142955 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:49.143425 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:49.143437 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.144529 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.144535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:49.146192 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:49.146208 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.146941 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.146948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:49.147145 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:49.147888 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:49.147910 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:49.149110 (+ 1200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:49.153020 (+ 3910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:49.153025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:49.153029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:49.153559 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:49.153951 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:49.153955 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:49.153959 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:49.154121 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:49.154716 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:49.154720 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:49.154885 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 06:50:49.154998 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 06:50:49.155261 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":803,"threads_started":1}
I20260810 06:50:49.155731 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:49.140507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:57949 (local address 127.22.11.190:36439)
0810 06:50:49.140699 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:49.140705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:49.141860 (+ 1155us) server_negotiation.cc:407] Connection header received
0810 06:50:49.142063 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:49.142068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:49.142150 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:49.142252 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:49.143571 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.144352 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.146337 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.146742 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.146857 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:49.149336 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:49.149345 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:49.149350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:49.149391 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:49.152899 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:49.153669 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:49.153674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:49.153676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:49.153839 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:49.154219 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:49.154223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:49.154225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:49.154578 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:49.154728 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:49.155171 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 06:50:49.155387 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:50:49.163201 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 06:50:49 dist-test-slave-f7th krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344649, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 06:50:49 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:50:49 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:50:49.650430 3783 heartbeater.cc:503] Master 127.22.11.190:36439 was elected leader, sending a full tablet report...
I20260810 06:50:50.156405 4029 heartbeater.cc:503] Master 127.22.11.190:36439 was elected leader, sending a full tablet report...
W20260810 06:50:50.237730 4046 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.026s user 0.001s sys 0.018s
W20260810 06:50:50.238929 3346 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:49.206392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59280 (local address 127.22.11.190:36439)
0810 06:50:49.206620 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:49.206625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:49.206733 (+ 108us) server_negotiation.cc:407] Connection header received
0810 06:50:49.206905 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:49.206910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:49.206981 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:49.207073 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:49.208313 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.209081 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.210282 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:49.210693 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:49.210786 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.238605 (+1027819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59280: BlockingRecv error: recv got EOF from 127.0.0.1:59280 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 10 06:50:50 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344649, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:50.261322 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.242950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.22.11.190:36439)
0810 06:50:50.243108 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.243113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.243438 (+ 325us) server_negotiation.cc:407] Connection header received
0810 06:50:50.243582 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.243586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.243675 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.243761 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:50.245028 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.245844 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.247453 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.248116 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.248238 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.254355 (+ 6117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:50.254362 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:50.254367 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:50.254408 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:50.257521 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:50.258468 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:50.258475 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:50.258479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:50.258665 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:50.259192 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:50.259198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:50.259201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:50.259613 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:50.259939 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.260827 (+ 888us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.261068 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:50:50.265695 3166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:50.274250 3166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:50.311796 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.289542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:40647 (local address 127.0.0.1:52752)
0810 06:50:50.296524 (+ 6982us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.296575 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.296784 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.303593 (+ 6809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.303601 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.303635 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:50.304120 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.304134 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.306269 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.306276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.308801 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.308819 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.310789 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.310795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.311030 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.311100 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.311224 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.311411 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6674,"threads_started":1}
I20260810 06:50:50.313180 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.289719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.22.11.130:40647)
0810 06:50:50.302949 (+ 13230us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.302959 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.302989 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:50:50.303102 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.303110 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.303363 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.303563 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:50.304361 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.306102 (+ 1741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.309023 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.310539 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.312354 (+ 1815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.312597 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.312689 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.312863 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13060,"thread_start_us":12929,"threads_started":1}
I20260810 06:50:50.321620 3661 tablet_service.cc:1467] Processing CreateTablet for tablet 79246e1558704a8f971f4d261a33000c (DEFAULT_TABLE table=test-table [id=f45fcfd433664b1bbda8741f7f81329c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:50.323256 3661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79246e1558704a8f971f4d261a33000c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.323482 4080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.289704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39037 (local address 127.0.0.1:50484)
0810 06:50:50.312072 (+ 22368us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.312092 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.312195 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.317737 (+ 5542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.317742 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.317766 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:50.317974 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.317985 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.319743 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.319749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.321113 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.321126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.322767 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.322773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.322949 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.322983 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.323075 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.323196 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22220,"mutex_wait_us":78,"thread_start_us":6363,"threads_started":1}
I20260810 06:50:50.323531 4084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.290293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.22.11.131:39037)
0810 06:50:50.317137 (+ 26844us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.317145 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.317174 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:50:50.317273 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.317280 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.317507 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.317740 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:50.318157 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.319556 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.321307 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.322617 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.322735 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.322927 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.323120 (+ 193us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.323273 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26668,"thread_start_us":26469,"threads_started":1}
I20260810 06:50:50.324493 4086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.291523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36015 (local address 127.0.0.1:51976)
0810 06:50:50.316278 (+ 24755us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.316297 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.316438 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.317156 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.317161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.317183 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:50.317626 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.317639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.319591 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.319596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.321711 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.321726 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.323740 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.323746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.323899 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.323931 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.324036 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.324155 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24629,"mutex_wait_us":1,"thread_start_us":24455,"threads_started":1}
I20260810 06:50:50.324527 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.292138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51976 (local address 127.22.11.129:36015)
0810 06:50:50.315777 (+ 23639us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.315786 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.316560 (+ 774us) server_negotiation.cc:407] Connection header received
0810 06:50:50.316697 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.316707 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.316955 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.317156 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:50.317823 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.319451 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.321898 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.323278 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.323410 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.323596 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.324090 (+ 494us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.324252 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23453,"thread_start_us":23303,"threads_started":1}
I20260810 06:50:50.326681 3914 tablet_service.cc:1467] Processing CreateTablet for tablet 79246e1558704a8f971f4d261a33000c (DEFAULT_TABLE table=test-table [id=f45fcfd433664b1bbda8741f7f81329c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:50.328007 3914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79246e1558704a8f971f4d261a33000c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.330696 3416 tablet_service.cc:1467] Processing CreateTablet for tablet 79246e1558704a8f971f4d261a33000c (DEFAULT_TABLE table=test-table [id=f45fcfd433664b1bbda8741f7f81329c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:50.332027 3416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79246e1558704a8f971f4d261a33000c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.337067 4091 tablet_bootstrap.cc:492] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Bootstrap starting.
I20260810 06:50:50.339612 4092 tablet_bootstrap.cc:492] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Bootstrap starting.
I20260810 06:50:50.340873 4091 tablet_bootstrap.cc:654] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:50.341984 4091 log.cc:824] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:50.343554 4092 tablet_bootstrap.cc:654] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:50.344794 4092 log.cc:824] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:50.348795 4099 tablet_bootstrap.cc:492] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Bootstrap starting.
I20260810 06:50:50.349850 4092 tablet_bootstrap.cc:492] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: No bootstrap required, opened a new log
I20260810 06:50:50.350117 4092 ts_tablet_manager.cc:1397] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 06:50:50.350950 4091 tablet_bootstrap.cc:492] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: No bootstrap required, opened a new log
I20260810 06:50:50.351228 4091 ts_tablet_manager.cc:1397] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260810 06:50:50.355455 4099 tablet_bootstrap.cc:654] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:50.356724 4099 log.cc:824] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:50.359014 4099 tablet_bootstrap.cc:492] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: No bootstrap required, opened a new log
I20260810 06:50:50.359242 4099 ts_tablet_manager.cc:1397] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260810 06:50:50.360561 4092 raft_consensus.cc:348] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.361035 4092 raft_consensus.cc:374] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:50.361141 4092 raft_consensus.cc:729] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98789c264f074fdf85c974e311e05fb5, State: Initialized, Role: FOLLOWER
I20260810 06:50:50.361539 4091 raft_consensus.cc:348] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.361769 4092 consensus_queue.cc:260] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.362010 4091 raft_consensus.cc:374] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:50.362114 4091 raft_consensus.cc:729] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1edcf5cefdf49fdaee772112fae5d54, State: Initialized, Role: FOLLOWER
I20260810 06:50:50.362708 4091 consensus_queue.cc:260] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.364473 4091 ts_tablet_manager.cc:1428] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260810 06:50:50.366227 4092 ts_tablet_manager.cc:1428] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260810 06:50:50.368407 4099 raft_consensus.cc:348] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.368790 4099 raft_consensus.cc:374] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:50.368906 4099 raft_consensus.cc:729] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03f5db71d114bf4b2ec97c40ad7d62f, State: Initialized, Role: FOLLOWER
I20260810 06:50:50.369459 4099 consensus_queue.cc:260] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.371181 4099 ts_tablet_manager.cc:1428] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
W20260810 06:50:50.380581 3538 tablet.cc:2369] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:50:50.395901 3784 tablet.cc:2369] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:50:50.402980 4034 tablet.cc:2369] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:50:50.448063 4105 raft_consensus.cc:482] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:50.448375 4105 raft_consensus.cc:504] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.450270 4105 leader_election.cc:288] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037), f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015)
I20260810 06:50:50.457656 4084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.450696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:46861 (local address 127.22.11.131:39037)
0810 06:50:50.450899 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.450906 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.451782 (+ 876us) server_negotiation.cc:407] Connection header received
0810 06:50:50.452030 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.452035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.452102 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.452182 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:50.452919 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.454007 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.455917 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.456721 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.456864 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.456938 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.457301 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.457416 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:50:50.457789 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.450798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:45313 (local address 127.22.11.129:36015)
0810 06:50:50.451018 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.451024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.451782 (+ 758us) server_negotiation.cc:407] Connection header received
0810 06:50:50.451971 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.451976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.452067 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.452161 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:50.452893 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.453983 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.455938 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.456921 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.457042 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.457106 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.457426 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.457566 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:50:50.457856 4109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.450662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39037 (local address 127.22.11.130:46861)
0810 06:50:50.451677 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.451692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.451879 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.452187 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.452191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.452231 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:50.452783 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.452792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.454186 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.454192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.455769 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.455781 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.456910 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.456916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.457107 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.457168 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.457271 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.457392 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":40,"spinlock_wait_cycles":56832,"thread_start_us":725,"threads_started":1}
I20260810 06:50:50.457878 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.450554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36015 (local address 127.22.11.130:45313)
0810 06:50:50.451671 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.451687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.451832 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.452186 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.452191 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.452220 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:50.452744 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.452758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.454135 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.454138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.455763 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.455776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.457105 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.457111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.457283 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.457312 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.457386 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.457484 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":25,"thread_start_us":699,"threads_started":1}
I20260810 06:50:50.458972 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79246e1558704a8f971f4d261a33000c" candidate_uuid: "c1edcf5cefdf49fdaee772112fae5d54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98789c264f074fdf85c974e311e05fb5" is_pre_election: true
I20260810 06:50:50.458967 3447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79246e1558704a8f971f4d261a33000c" candidate_uuid: "c1edcf5cefdf49fdaee772112fae5d54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" is_pre_election: true
I20260810 06:50:50.459429 3447 raft_consensus.cc:2454] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1edcf5cefdf49fdaee772112fae5d54 in term 0.
I20260810 06:50:50.459425 3944 raft_consensus.cc:2454] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1edcf5cefdf49fdaee772112fae5d54 in term 0.
I20260810 06:50:50.459995 3573 leader_election.cc:302] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1edcf5cefdf49fdaee772112fae5d54, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:50.460359 4105 raft_consensus.cc:2781] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:50.460515 4105 raft_consensus.cc:482] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:50.460640 4105 raft_consensus.cc:3037] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:50.462874 4105 raft_consensus.cc:504] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.463390 4105 leader_election.cc:288] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [CANDIDATE]: Term 1 election: Requested vote from peers 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037), f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015)
I20260810 06:50:50.463781 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79246e1558704a8f971f4d261a33000c" candidate_uuid: "c1edcf5cefdf49fdaee772112fae5d54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98789c264f074fdf85c974e311e05fb5"
I20260810 06:50:50.463838 3447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79246e1558704a8f971f4d261a33000c" candidate_uuid: "c1edcf5cefdf49fdaee772112fae5d54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f"
I20260810 06:50:50.463958 3944 raft_consensus.cc:3037] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:50.464022 3447 raft_consensus.cc:3037] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:50.466243 3447 raft_consensus.cc:2454] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1edcf5cefdf49fdaee772112fae5d54 in term 1.
I20260810 06:50:50.466493 3944 raft_consensus.cc:2454] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1edcf5cefdf49fdaee772112fae5d54 in term 1.
I20260810 06:50:50.466583 3573 leader_election.cc:302] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1edcf5cefdf49fdaee772112fae5d54, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:50.466874 4105 raft_consensus.cc:2781] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:50.467267 4105 raft_consensus.cc:686] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 1 LEADER]: Becoming Leader. State: Replica: c1edcf5cefdf49fdaee772112fae5d54, State: Running, Role: LEADER
I20260810 06:50:50.467690 4105 consensus_queue.cc:237] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } }
I20260810 06:50:50.475451 3166 catalog_manager.cc:5495] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130). New cstate: current_term: 1 leader_uuid: "c1edcf5cefdf49fdaee772112fae5d54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } health_report { overall_health: UNKNOWN } } }
I20260810 06:50:50.518004 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.512954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52754 (local address 127.22.11.130:40647)
0810 06:50:50.513165 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.513170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.513187 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:50:50.513380 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.513384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.513461 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.513587 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:50.513954 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.514657 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.516025 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.516443 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.516534 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.516934 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:50.517447 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:50.517559 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.517681 (+ 122us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.517743 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 06:50:50.526386 3944 raft_consensus.cc:1261] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Refusing update from remote peer c1edcf5cefdf49fdaee772112fae5d54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:50.526566 3447 raft_consensus.cc:1261] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Refusing update from remote peer c1edcf5cefdf49fdaee772112fae5d54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:50.527316 4105 consensus_queue.cc:1035] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:50.527590 4110 consensus_queue.cc:1035] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:50.541522 4117 mvcc.cc:205] Tried to move back new op lower bound from 7316867688545726464 to 7316867688318853120. Current Snapshot: MvccSnapshot[applied={T|T < 7316867688545726464}]
I20260810 06:50:50.556668 3166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:50.556882 3166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:50.560305 3166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Sending DeleteTablet for 3 replicas of tablet 79246e1558704a8f971f4d261a33000c
I20260810 06:50:50.561252 3661 tablet_service.cc:1514] Processing DeleteTablet for tablet 79246e1558704a8f971f4d261a33000c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:50 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52752
I20260810 06:50:50.561443 3914 tablet_service.cc:1514] Processing DeleteTablet for tablet 79246e1558704a8f971f4d261a33000c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:50 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50484
I20260810 06:50:50.562358 3416 tablet_service.cc:1514] Processing DeleteTablet for tablet 79246e1558704a8f971f4d261a33000c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:50 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:51976
I20260810 06:50:50.563226 3166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:50.564095 3166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:50.570927 4135 tablet_replica.cc:330] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: stopping tablet replica
I20260810 06:50:50.571457 4135 raft_consensus.cc:2229] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:50.573267 4135 raft_consensus.cc:2258] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:50.573472 3660 tablet_service.cc:1467] Processing CreateTablet for tablet 8534a793397044508c3cf45744a9a03a (DEFAULT_TABLE table=test-table [id=a56ff4b0209b4b2c96c0cf3a8a985b6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:50.574096 3660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8534a793397044508c3cf45744a9a03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.574304 4134 tablet_replica.cc:330] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: stopping tablet replica
I20260810 06:50:50.575191 4133 tablet_replica.cc:330] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: stopping tablet replica
I20260810 06:50:50.575620 4133 raft_consensus.cc:2229] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:50.575872 4134 raft_consensus.cc:2229] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:50.576370 4134 raft_consensus.cc:2258] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:50.576380 4133 raft_consensus.cc:2258] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:50.577669 3913 tablet_service.cc:1467] Processing CreateTablet for tablet 8534a793397044508c3cf45744a9a03a (DEFAULT_TABLE table=test-table [id=a56ff4b0209b4b2c96c0cf3a8a985b6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:50.577836 3413 tablet_service.cc:1467] Processing CreateTablet for tablet 8534a793397044508c3cf45744a9a03a (DEFAULT_TABLE table=test-table [id=a56ff4b0209b4b2c96c0cf3a8a985b6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:50.578084 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8534a793397044508c3cf45744a9a03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.578248 3413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8534a793397044508c3cf45744a9a03a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.583397 4091 tablet_bootstrap.cc:492] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: Bootstrap starting.
I20260810 06:50:50.584174 4092 tablet_bootstrap.cc:492] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: Bootstrap starting.
I20260810 06:50:50.585065 4091 tablet_bootstrap.cc:654] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:50.585458 4133 ts_tablet_manager.cc:1905] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:50.585721 4092 tablet_bootstrap.cc:654] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:50.587013 4099 tablet_bootstrap.cc:492] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: Bootstrap starting.
I20260810 06:50:50.588523 4099 tablet_bootstrap.cc:654] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:50.590855 4135 ts_tablet_manager.cc:1905] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:50.592530 4133 ts_tablet_manager.cc:1918] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:50.592715 4133 log.cc:1196] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/79246e1558704a8f971f4d261a33000c
I20260810 06:50:50.593241 4133 ts_tablet_manager.cc:1939] T 79246e1558704a8f971f4d261a33000c P c1edcf5cefdf49fdaee772112fae5d54: Deleting consensus metadata
I20260810 06:50:50.595093 3138 catalog_manager.cc:4841] TS c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647): tablet 79246e1558704a8f971f4d261a33000c (table test-table [id=f45fcfd433664b1bbda8741f7f81329c]) successfully deleted
I20260810 06:50:50.596766 4134 ts_tablet_manager.cc:1905] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:50.599436 4099 tablet_bootstrap.cc:492] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: No bootstrap required, opened a new log
I20260810 06:50:50.599586 4099 ts_tablet_manager.cc:1397] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.007s
I20260810 06:50:50.600461 4099 raft_consensus.cc:348] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.600709 4099 raft_consensus.cc:374] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:50.600819 4099 raft_consensus.cc:729] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03f5db71d114bf4b2ec97c40ad7d62f, State: Initialized, Role: FOLLOWER
I20260810 06:50:50.601166 4099 consensus_queue.cc:260] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.601713 4092 tablet_bootstrap.cc:492] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: No bootstrap required, opened a new log
I20260810 06:50:50.601871 4092 ts_tablet_manager.cc:1397] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.004s
I20260810 06:50:50.602788 4135 ts_tablet_manager.cc:1918] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:50.602779 4092 raft_consensus.cc:348] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.603030 4092 raft_consensus.cc:374] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:50.603050 4135 log.cc:1196] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/79246e1558704a8f971f4d261a33000c
I20260810 06:50:50.603142 4092 raft_consensus.cc:729] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98789c264f074fdf85c974e311e05fb5, State: Initialized, Role: FOLLOWER
I20260810 06:50:50.603415 4092 consensus_queue.cc:260] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.603839 4135 ts_tablet_manager.cc:1939] T 79246e1558704a8f971f4d261a33000c P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting consensus metadata
I20260810 06:50:50.603947 4092 ts_tablet_manager.cc:1428] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:50:50.603960 4099 ts_tablet_manager.cc:1428] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:50:50.605430 4134 ts_tablet_manager.cc:1918] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:50.605587 4134 log.cc:1196] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/79246e1558704a8f971f4d261a33000c
I20260810 06:50:50.605890 3132 catalog_manager.cc:4841] TS f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015): tablet 79246e1558704a8f971f4d261a33000c (table test-table [id=f45fcfd433664b1bbda8741f7f81329c]) successfully deleted
I20260810 06:50:50.606120 4134 ts_tablet_manager.cc:1939] T 79246e1558704a8f971f4d261a33000c P 98789c264f074fdf85c974e311e05fb5: Deleting consensus metadata
I20260810 06:50:50.607888 4091 tablet_bootstrap.cc:492] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: No bootstrap required, opened a new log
I20260810 06:50:50.608050 4091 ts_tablet_manager.cc:1397] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.000s
I20260810 06:50:50.608525 3136 catalog_manager.cc:4841] TS 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037): tablet 79246e1558704a8f971f4d261a33000c (table test-table [id=f45fcfd433664b1bbda8741f7f81329c]) successfully deleted
I20260810 06:50:50.608865 4091 raft_consensus.cc:348] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.609102 4091 raft_consensus.cc:374] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:50.609206 4091 raft_consensus.cc:729] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1edcf5cefdf49fdaee772112fae5d54, State: Initialized, Role: FOLLOWER
I20260810 06:50:50.609478 4091 consensus_queue.cc:260] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.610062 4091 ts_tablet_manager.cc:1428] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:50:50.882350 4104 raft_consensus.cc:482] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:50.882647 4104 raft_consensus.cc:504] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.884826 4104 leader_election.cc:288] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015), c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647)
I20260810 06:50:50.892830 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.885231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:40647 (local address 127.22.11.131:59927)
0810 06:50:50.886464 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.886482 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.886707 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.887191 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.887196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.887223 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:50.887590 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.887604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.888877 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.888881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.890310 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.890323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.891940 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.891947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.892132 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.892169 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.892281 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.892413 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"spinlock_wait_cycles":27136,"thread_start_us":901,"threads_started":1}
I20260810 06:50:50.892828 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.885933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:59927 (local address 127.22.11.130:40647)
0810 06:50:50.886099 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.886105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.886569 (+ 464us) server_negotiation.cc:407] Connection header received
0810 06:50:50.886908 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.886915 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.887044 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.887138 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:50.887794 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.888734 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.890469 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.891681 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.891843 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.891918 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.892325 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.892459 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 06:50:50.892901 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.886145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:51061 (local address 127.22.11.129:36015)
0810 06:50:50.886316 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.886322 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.886340 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:50:50.886414 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.886417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.886477 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.886562 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:50.887245 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.888148 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.890719 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.891914 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.892084 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.892154 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.892510 (+ 356us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.892668 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:50:50.893545 4140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.885047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36015 (local address 127.22.11.131:51061)
0810 06:50:50.885733 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.885756 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.885916 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.886605 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.886610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.886638 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:50.887063 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.887077 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.888302 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.888307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.890536 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.890553 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.892148 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.892154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.892317 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.892358 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.893115 (+ 757us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.893208 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"spinlock_wait_cycles":1280,"thread_start_us":367,"threads_started":1}
I20260810 06:50:50.894201 3447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8534a793397044508c3cf45744a9a03a" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" is_pre_election: true
I20260810 06:50:50.894279 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8534a793397044508c3cf45744a9a03a" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54" is_pre_election: true
I20260810 06:50:50.894486 3447 raft_consensus.cc:2454] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 0.
I20260810 06:50:50.894731 3703 raft_consensus.cc:2454] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 0.
I20260810 06:50:50.895004 3818 leader_election.cc:302] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98789c264f074fdf85c974e311e05fb5, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:50.895440 4104 raft_consensus.cc:2781] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:50.895558 4104 raft_consensus.cc:482] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:50.895648 4104 raft_consensus.cc:3037] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:50.897877 4104 raft_consensus.cc:504] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.898448 4104 leader_election.cc:288] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 election: Requested vote from peers f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015), c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647)
I20260810 06:50:50.898978 3447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8534a793397044508c3cf45744a9a03a" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f"
I20260810 06:50:50.898983 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8534a793397044508c3cf45744a9a03a" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54"
I20260810 06:50:50.899191 3703 raft_consensus.cc:3037] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:50.899180 3447 raft_consensus.cc:3037] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:50.902204 3447 raft_consensus.cc:2454] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 1.
I20260810 06:50:50.902568 3703 raft_consensus.cc:2454] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 1.
I20260810 06:50:50.902629 3818 leader_election.cc:302] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98789c264f074fdf85c974e311e05fb5, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:50.902907 4104 raft_consensus.cc:2781] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:50.903371 4104 raft_consensus.cc:686] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 1 LEADER]: Becoming Leader. State: Replica: 98789c264f074fdf85c974e311e05fb5, State: Running, Role: LEADER
I20260810 06:50:50.903770 4104 consensus_queue.cc:237] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:50.907250 3167 catalog_manager.cc:5495] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98789c264f074fdf85c974e311e05fb5 (127.22.11.131). New cstate: current_term: 1 leader_uuid: "98789c264f074fdf85c974e311e05fb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } health_report { overall_health: HEALTHY } } }
Aug 10 06:50:50 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344647, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.190@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:50.985096 4155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.970794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:36439 (local address 127.0.0.1:49962)
0810 06:50:50.971663 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:50.971684 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:50.971810 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:50.972151 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:50.972157 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:50.972870 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:50.973124 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.973133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.974558 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.974562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.975510 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:50.975521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.976432 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.976436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:50.976560 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.977191 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:50.977208 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:50.979290 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:50.982756 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:50.982767 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:50.982773 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:50.983230 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:50.983581 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:50.983585 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:50.983587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:50.983726 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:50.984287 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:50.984293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:50.984438 (+ 145us) client_negotiation.cc:769] Sending connection context
0810 06:50:50.984587 (+ 149us) client_negotiation.cc:240] Negotiation successful
0810 06:50:50.984814 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":570,"threads_started":1}
I20260810 06:50:50.985371 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:50.970825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.22.11.190:36439)
0810 06:50:50.971456 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:50.971462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:50.971760 (+ 298us) server_negotiation.cc:407] Connection header received
0810 06:50:50.971915 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:50.971920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:50.972004 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:50.972111 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:50.973298 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.974429 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.975644 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:50.976282 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:50.976408 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:50.979488 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:50.979494 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:50.979498 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:50.979530 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:50.982628 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:50.983333 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:50.983338 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:50.983341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:50.983478 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:50.983817 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:50.983822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:50.983826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:50.984173 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:50.984313 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:50.984730 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:50:50.984962 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":295,"threads_started":1}
I20260810 06:50:50.990103 3167 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:49962:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 06:50:50.991092 3167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:50.997466 3413 tablet_service.cc:1467] Processing CreateTablet for tablet fbea5b48b7b24984914be34a37178e9d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5dd3cd73f98549319b0842cf2378bd71]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:50:50.997627 3660 tablet_service.cc:1467] Processing CreateTablet for tablet fbea5b48b7b24984914be34a37178e9d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5dd3cd73f98549319b0842cf2378bd71]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:50:50.997870 3913 tablet_service.cc:1467] Processing CreateTablet for tablet fbea5b48b7b24984914be34a37178e9d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5dd3cd73f98549319b0842cf2378bd71]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 06:50:50.998075 3413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbea5b48b7b24984914be34a37178e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.998157 3660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbea5b48b7b24984914be34a37178e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:50.998355 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbea5b48b7b24984914be34a37178e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:51.003700 4099 tablet_bootstrap.cc:492] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: Bootstrap starting.
I20260810 06:50:51.004854 4091 tablet_bootstrap.cc:492] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: Bootstrap starting.
I20260810 06:50:51.005115 4099 tablet_bootstrap.cc:654] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.005332 4092 tablet_bootstrap.cc:492] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: Bootstrap starting.
I20260810 06:50:51.006412 4091 tablet_bootstrap.cc:654] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.006888 4092 tablet_bootstrap.cc:654] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.007371 4099 tablet_bootstrap.cc:492] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: No bootstrap required, opened a new log
I20260810 06:50:51.007536 4099 ts_tablet_manager.cc:1397] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:50:51.008282 4099 raft_consensus.cc:348] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.008543 4099 raft_consensus.cc:374] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.008667 4091 tablet_bootstrap.cc:492] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: No bootstrap required, opened a new log
I20260810 06:50:51.008863 4091 ts_tablet_manager.cc:1397] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260810 06:50:51.008859 4099 raft_consensus.cc:729] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03f5db71d114bf4b2ec97c40ad7d62f, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.009135 4099 consensus_queue.cc:260] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.009675 4092 tablet_bootstrap.cc:492] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: No bootstrap required, opened a new log
I20260810 06:50:51.009689 4099 ts_tablet_manager.cc:1428] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 06:50:51.009591 4107 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.009893 4092 ts_tablet_manager.cc:1397] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 06:50:51.010021 4107 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:50:51.010449 4091 raft_consensus.cc:348] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.010560 4092 raft_consensus.cc:348] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.010691 4091 raft_consensus.cc:374] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.010761 4091 raft_consensus.cc:729] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1edcf5cefdf49fdaee772112fae5d54, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.010778 4092 raft_consensus.cc:374] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.010846 4092 raft_consensus.cc:729] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98789c264f074fdf85c974e311e05fb5, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.011060 4092 consensus_queue.cc:260] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.011039 4091 consensus_queue.cc:260] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.011444 4104 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.011444 4110 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.011721 4110 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:50:51.011729 4104 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:50:51.012118 4092 ts_tablet_manager.cc:1428] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 06:50:51.012151 4091 ts_tablet_manager.cc:1428] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:51.051465 4107 raft_consensus.cc:482] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:51.051663 4107 raft_consensus.cc:504] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.053189 4107 leader_election.cc:288] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647), 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037)
I20260810 06:50:51.059757 4078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.053543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:51065 (local address 127.22.11.130:40647)
0810 06:50:51.053728 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.053735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.054183 (+ 448us) server_negotiation.cc:407] Connection header received
0810 06:50:51.054568 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.054574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.054652 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.054734 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:51.055432 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.056157 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.058091 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.058796 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.058938 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.059009 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.059379 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.059517 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:50:51.060029 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.053510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:42369 (local address 127.22.11.131:39037)
0810 06:50:51.054331 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.054337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.054613 (+ 276us) server_negotiation.cc:407] Connection header received
0810 06:50:51.054760 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.054765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.054838 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.054944 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:50:51.055506 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.056273 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.057994 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.058788 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.058938 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.059011 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.059340 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.059473 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":596,"threads_started":1}
I20260810 06:50:51.059868 4162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.053481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:40647 (local address 127.22.11.129:51065)
0810 06:50:51.054081 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:51.054101 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:51.054308 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:51.054756 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:51.054761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:51.054788 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:51.055252 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.055266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.056289 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.056294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.057930 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.057947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.059066 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.059072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.059230 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.059253 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:50:51.059339 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:50:51.059463 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":34,"thread_start_us":359,"threads_started":1}
I20260810 06:50:51.060032 4164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.053556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39037 (local address 127.22.11.129:42369)
0810 06:50:51.054512 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:51.054527 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:51.054651 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:51.054962 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:51.054966 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:51.054989 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:50:51.055327 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.055338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.056398 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.056402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.057852 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.057864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.058999 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.059004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.059192 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.059221 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:50:51.059292 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:50:51.059420 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":39,"thread_start_us":724,"threads_started":1}
I20260810 06:50:51.060799 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea5b48b7b24984914be34a37178e9d" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54" is_pre_election: true
I20260810 06:50:51.060813 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea5b48b7b24984914be34a37178e9d" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98789c264f074fdf85c974e311e05fb5" is_pre_election: true
I20260810 06:50:51.061020 3703 raft_consensus.cc:2454] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 0.
I20260810 06:50:51.061021 3944 raft_consensus.cc:2454] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 0.
I20260810 06:50:51.061465 3330 leader_election.cc:302] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98789c264f074fdf85c974e311e05fb5, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:51.061822 4107 raft_consensus.cc:2781] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:51.061932 4107 raft_consensus.cc:482] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:51.062043 4107 raft_consensus.cc:3037] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.064088 4107 raft_consensus.cc:504] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.064582 4107 leader_election.cc:288] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 election: Requested vote from peers c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647), 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037)
I20260810 06:50:51.065006 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea5b48b7b24984914be34a37178e9d" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54"
I20260810 06:50:51.065023 3944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbea5b48b7b24984914be34a37178e9d" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98789c264f074fdf85c974e311e05fb5"
I20260810 06:50:51.065181 3944 raft_consensus.cc:3037] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.065224 3703 raft_consensus.cc:3037] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.067184 3944 raft_consensus.cc:2454] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 1.
I20260810 06:50:51.067217 3703 raft_consensus.cc:2454] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 1.
I20260810 06:50:51.067533 3330 leader_election.cc:302] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98789c264f074fdf85c974e311e05fb5, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:51.067813 4107 raft_consensus.cc:2781] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:51.068199 4107 raft_consensus.cc:686] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 LEADER]: Becoming Leader. State: Replica: f03f5db71d114bf4b2ec97c40ad7d62f, State: Running, Role: LEADER
I20260810 06:50:51.068531 4107 consensus_queue.cc:237] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.070226 4165 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: TxnStatusTablet state changed. Reason: New leader f03f5db71d114bf4b2ec97c40ad7d62f. Latest consensus state: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.070537 4165 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:50:51.071228 4168 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 06:50:51.071436 4168 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 06:50:51.071903 3166 catalog_manager.cc:5495] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f reported cstate change: term changed from 0 to 1, leader changed from <none> to f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129). New cstate: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } health_report { overall_health: UNKNOWN } } }
I20260810 06:50:51.088868 4155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.083035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36015 (local address 127.0.0.1:36628)
0810 06:50:51.083263 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:51.083284 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:51.083426 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:51.083882 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:51.083886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:51.083894 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:50:51.084045 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.084052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.085451 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.085455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.086399 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.086411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.087404 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.087408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.087542 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.087593 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.088446 (+ 853us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.088464 (+ 18us) client_negotiation.cc:769] Sending connection context
0810 06:50:51.088512 (+ 48us) client_negotiation.cc:240] Negotiation successful
0810 06:50:51.088637 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:50:51.088975 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.083114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.22.11.129:36015)
0810 06:50:51.083286 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.083291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.083383 (+ 92us) server_negotiation.cc:407] Connection header received
0810 06:50:51.083609 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.083615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.083727 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.083883 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:51.084161 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.085317 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.086547 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.087264 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.087398 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.087767 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.088353 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.088507 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.088565 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.088682 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:50:51.092895 3944 raft_consensus.cc:1261] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Refusing update from remote peer f03f5db71d114bf4b2ec97c40ad7d62f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.093108 3703 raft_consensus.cc:1261] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Refusing update from remote peer f03f5db71d114bf4b2ec97c40ad7d62f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.093693 4165 consensus_queue.cc:1035] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [LEADER]: Connected to new peer: Peer: permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.093990 4107 consensus_queue.cc:1035] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.097249 4110 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: TxnStatusTablet state changed. Reason: New leader f03f5db71d114bf4b2ec97c40ad7d62f. Latest consensus state: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.097513 4110 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:50:51.104749 4172 mvcc.cc:205] Tried to move back new op lower bound from 7316867690869805056 to 7316867690779918336. Current Snapshot: MvccSnapshot[applied={T|T < 7316867690869805056}]
I20260810 06:50:51.108685 4107 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.108938 4107 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:50:51.109885 4110 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.110122 4110 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P c1edcf5cefdf49fdaee772112fae5d54: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:50:51.118312 4104 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: TxnStatusTablet state changed. Reason: New leader f03f5db71d114bf4b2ec97c40ad7d62f. Latest consensus state: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.118629 4104 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:50:51.118592 4165 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.118834 4165 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P f03f5db71d114bf4b2ec97c40ad7d62f: This TxnStatusTablet replica's current role is: LEADER
I20260810 06:50:51.122217 4104 tablet_replica.cc:439] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } } }
I20260810 06:50:51.122447 4104 tablet_replica.cc:442] T fbea5b48b7b24984914be34a37178e9d P 98789c264f074fdf85c974e311e05fb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 06:50:51.126479 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.117025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.22.11.131:39037)
0810 06:50:51.117233 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.117241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.119941 (+ 2700us) server_negotiation.cc:407] Connection header received
0810 06:50:51.120262 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.120268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.120382 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.120515 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:51.121261 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.122399 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.124397 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.124853 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.124946 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.125384 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.125905 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.126015 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.126129 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.126210 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:50:51.149627 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.136999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:33629 (local address 127.22.11.129:36015)
0810 06:50:51.137191 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.137200 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.143199 (+ 5999us) server_negotiation.cc:407] Connection header received
0810 06:50:51.143363 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.143368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.143449 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.143557 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:51.144066 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.145199 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.147349 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.148021 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.148116 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.148796 (+ 680us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.148939 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.149004 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.149216 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.149301 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:50:51.150604 4192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.136916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36015 (local address 127.22.11.131:33629)
0810 06:50:51.143091 (+ 6175us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:51.143110 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:51.143242 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:51.143576 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:51.143580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:51.143592 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:50:51.143883 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.143894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.145358 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.145364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.147167 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.147186 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.148310 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.148317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.148485 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.148562 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.149084 (+ 522us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.149094 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 06:50:51.150036 (+ 942us) client_negotiation.cc:240] Negotiation successful
0810 06:50:51.150177 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6012,"thread_start_us":5862,"threads_started":1}
I20260810 06:50:51.163304 3703 raft_consensus.cc:1261] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Refusing update from remote peer 98789c264f074fdf85c974e311e05fb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.163463 3447 raft_consensus.cc:1261] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Refusing update from remote peer 98789c264f074fdf85c974e311e05fb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.164074 4143 consensus_queue.cc:1035] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.164922 4143 consensus_queue.cc:1035] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.213577 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.207002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:35919 (local address 127.22.11.131:39037)
0810 06:50:51.207160 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.207167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.207894 (+ 727us) server_negotiation.cc:407] Connection header received
0810 06:50:51.208077 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.208082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.208163 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.208236 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:51.208877 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.210027 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.211660 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.212196 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.212313 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.212883 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.213017 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.213109 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.213197 (+ 88us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.213277 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:50:51.213595 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.207087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:39037 (local address 127.22.11.129:35919)
0810 06:50:51.207786 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:51.207806 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:51.207948 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:51.208268 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:51.208273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:51.208285 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 06:50:51.208673 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.208687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.210144 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.210148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.211532 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:51.211544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.212472 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.212479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:51.212657 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.212723 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.213109 (+ 386us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.213118 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:50:51.213173 (+ 55us) client_negotiation.cc:240] Negotiation successful
0810 06:50:51.213270 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":334,"threads_started":1}
I20260810 06:50:51.259178 3165 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:51.259433 3165 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:51.263389 3165 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Sending DeleteTablet for 3 replicas of tablet 8534a793397044508c3cf45744a9a03a
I20260810 06:50:51.264247 3660 tablet_service.cc:1514] Processing DeleteTablet for tablet 8534a793397044508c3cf45744a9a03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52752
I20260810 06:50:51.264325 3913 tablet_service.cc:1514] Processing DeleteTablet for tablet 8534a793397044508c3cf45744a9a03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50484
I20260810 06:50:51.265163 3413 tablet_service.cc:1514] Processing DeleteTablet for tablet 8534a793397044508c3cf45744a9a03a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:51976
I20260810 06:50:51.288875 4210 tablet_replica.cc:330] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: stopping tablet replica
I20260810 06:50:51.289256 4210 raft_consensus.cc:2229] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:51.289834 4209 tablet_replica.cc:330] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: stopping tablet replica
I20260810 06:50:51.290180 4210 raft_consensus.cc:2258] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.290465 4209 raft_consensus.cc:2229] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:51.291301 4209 raft_consensus.cc:2258] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.292218 4210 ts_tablet_manager.cc:1905] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.293404 4209 ts_tablet_manager.cc:1905] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.295059 4207 tablet_replica.cc:330] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: stopping tablet replica
I20260810 06:50:51.295351 4207 raft_consensus.cc:2229] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:51.295635 4207 raft_consensus.cc:2258] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.296712 4207 ts_tablet_manager.cc:1905] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.301796 4209 ts_tablet_manager.cc:1918] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:51.303685 4207 ts_tablet_manager.cc:1918] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:51.303870 4207 log.cc:1196] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/8534a793397044508c3cf45744a9a03a
I20260810 06:50:51.304368 4207 ts_tablet_manager.cc:1939] T 8534a793397044508c3cf45744a9a03a P c1edcf5cefdf49fdaee772112fae5d54: Deleting consensus metadata
I20260810 06:50:51.305552 3138 catalog_manager.cc:4841] TS c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647): tablet 8534a793397044508c3cf45744a9a03a (table test-table [id=a56ff4b0209b4b2c96c0cf3a8a985b6d]) successfully deleted
I20260810 06:50:51.307767 4210 ts_tablet_manager.cc:1918] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:51.307986 4209 log.cc:1196] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/8534a793397044508c3cf45744a9a03a
I20260810 06:50:51.308542 4209 ts_tablet_manager.cc:1939] T 8534a793397044508c3cf45744a9a03a P 98789c264f074fdf85c974e311e05fb5: Deleting consensus metadata
I20260810 06:50:51.308728 4210 log.cc:1196] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/8534a793397044508c3cf45744a9a03a
I20260810 06:50:51.309265 4210 ts_tablet_manager.cc:1939] T 8534a793397044508c3cf45744a9a03a P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting consensus metadata
I20260810 06:50:51.311510 3132 catalog_manager.cc:4841] TS f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015): tablet 8534a793397044508c3cf45744a9a03a (table test-table [id=a56ff4b0209b4b2c96c0cf3a8a985b6d]) successfully deleted
I20260810 06:50:51.312459 3136 catalog_manager.cc:4841] TS 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037): tablet 8534a793397044508c3cf45744a9a03a (table test-table [id=a56ff4b0209b4b2c96c0cf3a8a985b6d]) successfully deleted
Aug 10 06:50:51 dist-test-slave-f7th krb5kdc[3085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 06:50:51 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:50:51 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:50:51.347723 4156 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:51.337383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49976 (local address 127.22.11.190:36439)
0810 06:50:51.337567 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.337574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.338052 (+ 478us) server_negotiation.cc:407] Connection header received
0810 06:50:51.338232 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.338239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.338323 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.338408 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:51.339700 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.340456 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.341713 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.342139 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.342248 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.347544 (+ 5296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49976: BlockingRecv error: recv got EOF from 127.0.0.1:49976 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 06:50:51 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM
I20260810 06:50:51.364817 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.351111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.22.11.190:36439)
0810 06:50:51.351275 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.351281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.351730 (+ 449us) server_negotiation.cc:407] Connection header received
0810 06:50:51.351937 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.351943 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.352014 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.352093 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:51.353361 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.354117 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.355424 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.355836 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.355943 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.359434 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:51.359440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:51.359443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:51.359484 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:51.362219 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:51.362792 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:51.362798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:51.362802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:51.362935 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:51.363282 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:51.363287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:51.363291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:51.363620 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:51.363868 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.364142 (+ 274us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.364348 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:50:51.368079 3165 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:51.368732 3165 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:51.375450 3413 tablet_service.cc:1467] Processing CreateTablet for tablet 88a4225512204f698524d7ce10c2e13a (DEFAULT_TABLE table=test-table [id=cb70656559be4b25a7ffac378c7e14a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:51.375624 3660 tablet_service.cc:1467] Processing CreateTablet for tablet 88a4225512204f698524d7ce10c2e13a (DEFAULT_TABLE table=test-table [id=cb70656559be4b25a7ffac378c7e14a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:51.375912 3413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a4225512204f698524d7ce10c2e13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:51.376106 3660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a4225512204f698524d7ce10c2e13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:51.381446 3913 tablet_service.cc:1467] Processing CreateTablet for tablet 88a4225512204f698524d7ce10c2e13a (DEFAULT_TABLE table=test-table [id=cb70656559be4b25a7ffac378c7e14a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:51.381947 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a4225512204f698524d7ce10c2e13a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:51.382382 4091 tablet_bootstrap.cc:492] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: Bootstrap starting.
I20260810 06:50:51.383958 4091 tablet_bootstrap.cc:654] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.391844 4099 tablet_bootstrap.cc:492] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: Bootstrap starting.
I20260810 06:50:51.393567 4099 tablet_bootstrap.cc:654] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.396311 4091 tablet_bootstrap.cc:492] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: No bootstrap required, opened a new log
I20260810 06:50:51.396508 4091 ts_tablet_manager.cc:1397] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260810 06:50:51.396727 4092 tablet_bootstrap.cc:492] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: Bootstrap starting.
I20260810 06:50:51.397475 4091 raft_consensus.cc:348] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.397868 4091 raft_consensus.cc:374] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.398028 4091 raft_consensus.cc:729] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1edcf5cefdf49fdaee772112fae5d54, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.398331 4092 tablet_bootstrap.cc:654] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.398360 4091 consensus_queue.cc:260] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.401649 4091 ts_tablet_manager.cc:1428] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260810 06:50:51.404642 4099 tablet_bootstrap.cc:492] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: No bootstrap required, opened a new log
I20260810 06:50:51.404847 4099 ts_tablet_manager.cc:1397] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.003s
I20260810 06:50:51.405691 4099 raft_consensus.cc:348] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.406026 4099 raft_consensus.cc:374] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.406172 4099 raft_consensus.cc:729] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03f5db71d114bf4b2ec97c40ad7d62f, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.406229 4092 tablet_bootstrap.cc:492] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: No bootstrap required, opened a new log
I20260810 06:50:51.406383 4092 ts_tablet_manager.cc:1397] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.003s
I20260810 06:50:51.406569 4099 consensus_queue.cc:260] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.407738 4099 ts_tablet_manager.cc:1428] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:50:51.407209 4092 raft_consensus.cc:348] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.408722 4092 raft_consensus.cc:374] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.408829 4092 raft_consensus.cc:729] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98789c264f074fdf85c974e311e05fb5, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.409065 4092 consensus_queue.cc:260] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.409581 4092 ts_tablet_manager.cc:1428] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 06:50:51.458267 4196 raft_consensus.cc:482] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:51.458552 4196 raft_consensus.cc:504] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.459470 4196 leader_election.cc:288] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647), 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037)
I20260810 06:50:51.460124 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a4225512204f698524d7ce10c2e13a" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54" is_pre_election: true
I20260810 06:50:51.460232 3943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a4225512204f698524d7ce10c2e13a" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98789c264f074fdf85c974e311e05fb5" is_pre_election: true
I20260810 06:50:51.460404 3703 raft_consensus.cc:2454] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 0.
I20260810 06:50:51.460529 3943 raft_consensus.cc:2454] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 0.
I20260810 06:50:51.461072 3330 leader_election.cc:302] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98789c264f074fdf85c974e311e05fb5, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:51.461467 4196 raft_consensus.cc:2781] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:51.461608 4196 raft_consensus.cc:482] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:51.461699 4196 raft_consensus.cc:3037] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.465126 4196 raft_consensus.cc:504] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.466166 4196 leader_election.cc:288] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 election: Requested vote from peers c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647), 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037)
I20260810 06:50:51.466305 3703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a4225512204f698524d7ce10c2e13a" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54"
I20260810 06:50:51.466507 3703 raft_consensus.cc:3037] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.467206 3943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a4225512204f698524d7ce10c2e13a" candidate_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98789c264f074fdf85c974e311e05fb5"
I20260810 06:50:51.467406 3943 raft_consensus.cc:3037] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.469487 3703 raft_consensus.cc:2454] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 1.
I20260810 06:50:51.469921 3331 leader_election.cc:302] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1edcf5cefdf49fdaee772112fae5d54, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:51.470286 4196 raft_consensus.cc:2781] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:51.470569 3943 raft_consensus.cc:2454] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f03f5db71d114bf4b2ec97c40ad7d62f in term 1.
I20260810 06:50:51.470746 4196 raft_consensus.cc:686] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 LEADER]: Becoming Leader. State: Replica: f03f5db71d114bf4b2ec97c40ad7d62f, State: Running, Role: LEADER
I20260810 06:50:51.471046 4196 consensus_queue.cc:237] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.473953 3165 catalog_manager.cc:5495] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f reported cstate change: term changed from 0 to 1, leader changed from <none> to f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129). New cstate: current_term: 1 leader_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } health_report { overall_health: UNKNOWN } } }
I20260810 06:50:51.502815 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.496793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.22.11.129:36015)
0810 06:50:51.496970 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.496976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.497040 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:50:51.497241 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.497247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.497337 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.497436 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:51.497831 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.498970 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.500881 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.501487 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.501636 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.502051 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.502193 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.502270 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.502410 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.502486 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:50:51.505945 3943 raft_consensus.cc:1261] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Refusing update from remote peer f03f5db71d114bf4b2ec97c40ad7d62f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.506158 3703 raft_consensus.cc:1261] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Refusing update from remote peer f03f5db71d114bf4b2ec97c40ad7d62f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.506860 4196 consensus_queue.cc:1035] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.507365 4165 consensus_queue.cc:1035] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [LEADER]: Connected to new peer: Peer: permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.522501 3165 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:51.522725 3165 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:51.529253 4175 mvcc.cc:205] Tried to move back new op lower bound from 7316867692562407424 to 7316867692428345344. Current Snapshot: MvccSnapshot[applied={T|T < 7316867692562407424}]
I20260810 06:50:51.531921 3165 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Sending DeleteTablet for 3 replicas of tablet 88a4225512204f698524d7ce10c2e13a
I20260810 06:50:51.533358 3660 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a4225512204f698524d7ce10c2e13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52752
I20260810 06:50:51.533668 4207 tablet_replica.cc:330] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: stopping tablet replica
I20260810 06:50:51.533865 4207 raft_consensus.cc:2229] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:51.534077 4207 raft_consensus.cc:2258] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.537312 4207 ts_tablet_manager.cc:1905] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.540856 3413 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a4225512204f698524d7ce10c2e13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:51976
I20260810 06:50:51.541157 4210 tablet_replica.cc:330] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: stopping tablet replica
I20260810 06:50:51.541355 4210 raft_consensus.cc:2229] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:51.541857 4210 raft_consensus.cc:2258] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.543069 4210 ts_tablet_manager.cc:1905] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.545768 3413 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a4225512204f698524d7ce10c2e13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:51976
I20260810 06:50:51.547079 3660 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a4225512204f698524d7ce10c2e13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52752
I20260810 06:50:51.547312 3913 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a4225512204f698524d7ce10c2e13a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50484
I20260810 06:50:51.547677 4209 tablet_replica.cc:330] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: stopping tablet replica
I20260810 06:50:51.547886 4209 raft_consensus.cc:2229] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:51.548144 4209 raft_consensus.cc:2258] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.549011 3166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:50:51.549743 3166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:50:51.552045 4207 ts_tablet_manager.cc:1918] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:51.552188 4207 log.cc:1196] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/88a4225512204f698524d7ce10c2e13a
I20260810 06:50:51.552588 4207 ts_tablet_manager.cc:1939] T 88a4225512204f698524d7ce10c2e13a P c1edcf5cefdf49fdaee772112fae5d54: Deleting consensus metadata
I20260810 06:50:51.553678 3138 catalog_manager.cc:4841] TS c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647): tablet 88a4225512204f698524d7ce10c2e13a (table test-table [id=cb70656559be4b25a7ffac378c7e14a6]) successfully deleted
W20260810 06:50:51.554590 3138 catalog_manager.cc:4808] TS c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647): delete failed for tablet 88a4225512204f698524d7ce10c2e13a because the tablet was not found. No further retry: Not found: Tablet not found: 88a4225512204f698524d7ce10c2e13a
I20260810 06:50:51.559150 3660 tablet_service.cc:1467] Processing CreateTablet for tablet 703dae1c194e4020a0cdb78e3f038263 (DEFAULT_TABLE table=test-table [id=7dea448b9ea3403eae258e5e5d002d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:51.559584 3660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703dae1c194e4020a0cdb78e3f038263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:51.560642 4210 ts_tablet_manager.cc:1918] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:51.560856 4210 log.cc:1196] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/88a4225512204f698524d7ce10c2e13a
I20260810 06:50:51.561352 4210 ts_tablet_manager.cc:1939] T 88a4225512204f698524d7ce10c2e13a P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting consensus metadata
I20260810 06:50:51.562919 3132 catalog_manager.cc:4841] TS f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015): tablet 88a4225512204f698524d7ce10c2e13a (table test-table [id=cb70656559be4b25a7ffac378c7e14a6]) successfully deleted
I20260810 06:50:51.563642 4209 ts_tablet_manager.cc:1905] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260810 06:50:51.564796 3132 catalog_manager.cc:4808] TS f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015): delete failed for tablet 88a4225512204f698524d7ce10c2e13a because the tablet was not found. No further retry: Not found: Tablet not found: 88a4225512204f698524d7ce10c2e13a
I20260810 06:50:51.566875 4091 tablet_bootstrap.cc:492] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: Bootstrap starting.
I20260810 06:50:51.568431 4091 tablet_bootstrap.cc:654] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.569223 3413 tablet_service.cc:1467] Processing CreateTablet for tablet 703dae1c194e4020a0cdb78e3f038263 (DEFAULT_TABLE table=test-table [id=7dea448b9ea3403eae258e5e5d002d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:51.569618 3413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703dae1c194e4020a0cdb78e3f038263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:51.573019 3913 tablet_service.cc:1467] Processing CreateTablet for tablet 703dae1c194e4020a0cdb78e3f038263 (DEFAULT_TABLE table=test-table [id=7dea448b9ea3403eae258e5e5d002d96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:50:51.574116 3913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703dae1c194e4020a0cdb78e3f038263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:51.580075 4099 tablet_bootstrap.cc:492] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: Bootstrap starting.
I20260810 06:50:51.581841 4099 tablet_bootstrap.cc:654] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.582065 4209 ts_tablet_manager.cc:1918] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:50:51.582216 4209 log.cc:1196] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/88a4225512204f698524d7ce10c2e13a
I20260810 06:50:51.582628 4209 ts_tablet_manager.cc:1939] T 88a4225512204f698524d7ce10c2e13a P 98789c264f074fdf85c974e311e05fb5: Deleting consensus metadata
I20260810 06:50:51.583541 4091 tablet_bootstrap.cc:492] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: No bootstrap required, opened a new log
I20260810 06:50:51.583715 4091 ts_tablet_manager.cc:1397] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.009s
I20260810 06:50:51.584481 4091 raft_consensus.cc:348] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.584779 4091 raft_consensus.cc:374] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.584867 4091 raft_consensus.cc:729] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1edcf5cefdf49fdaee772112fae5d54, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.587393 4091 consensus_queue.cc:260] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.588043 3136 catalog_manager.cc:4841] TS 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037): tablet 88a4225512204f698524d7ce10c2e13a (table test-table [id=cb70656559be4b25a7ffac378c7e14a6]) successfully deleted
I20260810 06:50:51.588292 4092 tablet_bootstrap.cc:492] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: Bootstrap starting.
I20260810 06:50:51.588809 4091 ts_tablet_manager.cc:1428] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 06:50:51.590024 4092 tablet_bootstrap.cc:654] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:51.605234 4092 tablet_bootstrap.cc:492] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: No bootstrap required, opened a new log
I20260810 06:50:51.605412 4092 ts_tablet_manager.cc:1397] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.008s
I20260810 06:50:51.605949 4099 tablet_bootstrap.cc:492] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: No bootstrap required, opened a new log
I20260810 06:50:51.606138 4099 ts_tablet_manager.cc:1397] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.007s
I20260810 06:50:51.606266 4092 raft_consensus.cc:348] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.606554 4092 raft_consensus.cc:374] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.606659 4092 raft_consensus.cc:729] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98789c264f074fdf85c974e311e05fb5, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.606980 4092 consensus_queue.cc:260] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.606999 4099 raft_consensus.cc:348] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.607275 4099 raft_consensus.cc:374] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:51.607368 4099 raft_consensus.cc:729] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f03f5db71d114bf4b2ec97c40ad7d62f, State: Initialized, Role: FOLLOWER
I20260810 06:50:51.607666 4099 consensus_queue.cc:260] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.608161 4092 ts_tablet_manager.cc:1428] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 06:50:51.612092 4099 ts_tablet_manager.cc:1428] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20260810 06:50:51.727419 4195 raft_consensus.cc:482] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:50:51.727653 4195 raft_consensus.cc:504] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.728415 4195 leader_election.cc:288] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647), f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015)
I20260810 06:50:51.729017 3702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703dae1c194e4020a0cdb78e3f038263" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54" is_pre_election: true
I20260810 06:50:51.729151 3446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703dae1c194e4020a0cdb78e3f038263" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" is_pre_election: true
I20260810 06:50:51.729279 3702 raft_consensus.cc:2454] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 0.
I20260810 06:50:51.729422 3446 raft_consensus.cc:2454] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 0.
I20260810 06:50:51.729943 3818 leader_election.cc:302] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98789c264f074fdf85c974e311e05fb5, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:51.730258 4195 raft_consensus.cc:2781] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:50:51.730377 4195 raft_consensus.cc:482] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:50:51.730479 4195 raft_consensus.cc:3037] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.734602 4195 raft_consensus.cc:504] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.735597 3702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703dae1c194e4020a0cdb78e3f038263" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1edcf5cefdf49fdaee772112fae5d54"
I20260810 06:50:51.735785 3702 raft_consensus.cc:3037] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.736020 3446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703dae1c194e4020a0cdb78e3f038263" candidate_uuid: "98789c264f074fdf85c974e311e05fb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f"
I20260810 06:50:51.736212 3446 raft_consensus.cc:3037] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:51.738593 3702 raft_consensus.cc:2454] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 1.
I20260810 06:50:51.739293 3446 raft_consensus.cc:2454] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98789c264f074fdf85c974e311e05fb5 in term 1.
I20260810 06:50:51.739733 3818 leader_election.cc:302] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98789c264f074fdf85c974e311e05fb5, f03f5db71d114bf4b2ec97c40ad7d62f; no voters:
I20260810 06:50:51.739818 4195 leader_election.cc:288] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [CANDIDATE]: Term 1 election: Requested vote from peers c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647), f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015)
I20260810 06:50:51.740062 4195 raft_consensus.cc:2781] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:51.740239 4195 raft_consensus.cc:686] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 1 LEADER]: Becoming Leader. State: Replica: 98789c264f074fdf85c974e311e05fb5, State: Running, Role: LEADER
I20260810 06:50:51.740511 4195 consensus_queue.cc:237] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } }
I20260810 06:50:51.748909 3165 catalog_manager.cc:5495] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98789c264f074fdf85c974e311e05fb5 (127.22.11.131). New cstate: current_term: 1 leader_uuid: "98789c264f074fdf85c974e311e05fb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98789c264f074fdf85c974e311e05fb5" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 39037 } health_report { overall_health: HEALTHY } } }
I20260810 06:50:51.787039 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.779655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.22.11.131:39037)
0810 06:50:51.781028 (+ 1373us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.781033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.781050 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:50:51.781107 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.781112 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.781198 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.781289 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:50:51.781808 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.783057 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.784942 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.785617 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.785735 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.786244 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:50:51.786388 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:50:51.786477 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.786613 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.786693 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1249,"thread_start_us":1118,"threads_started":1}
I20260810 06:50:51.797612 3446 raft_consensus.cc:1261] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Refusing update from remote peer 98789c264f074fdf85c974e311e05fb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.797776 3702 raft_consensus.cc:1261] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Refusing update from remote peer 98789c264f074fdf85c974e311e05fb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:50:51.798228 4195 consensus_queue.cc:1035] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f03f5db71d114bf4b2ec97c40ad7d62f" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.798538 4258 consensus_queue.cc:1035] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1edcf5cefdf49fdaee772112fae5d54" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 40647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:50:51.865903 3166 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:51.866124 3166 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:50:51.868865 3166 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0da5badb4a034d11b180f044322910e1: Sending DeleteTablet for 3 replicas of tablet 703dae1c194e4020a0cdb78e3f038263
I20260810 06:50:51.869585 3660 tablet_service.cc:1514] Processing DeleteTablet for tablet 703dae1c194e4020a0cdb78e3f038263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52752
I20260810 06:50:51.869848 3913 tablet_service.cc:1514] Processing DeleteTablet for tablet 703dae1c194e4020a0cdb78e3f038263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50484
I20260810 06:50:51.869843 3413 tablet_service.cc:1514] Processing DeleteTablet for tablet 703dae1c194e4020a0cdb78e3f038263 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:50:51 UTC) from {username='oryx', principal='oryx/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:51976
I20260810 06:50:51.869900 4207 tablet_replica.cc:330] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: stopping tablet replica
I20260810 06:50:51.870086 4207 raft_consensus.cc:2229] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:51.870115 4209 tablet_replica.cc:330] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: stopping tablet replica
I20260810 06:50:51.870123 4210 tablet_replica.cc:330] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: stopping tablet replica
I20260810 06:50:51.870278 4209 raft_consensus.cc:2229] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:50:51.870286 4210 raft_consensus.cc:2229] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:50:51.870291 4207 raft_consensus.cc:2258] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.870476 4210 raft_consensus.cc:2258] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.870590 4209 raft_consensus.cc:2258] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:50:51.871682 4210 ts_tablet_manager.cc:1905] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.871762 4207 ts_tablet_manager.cc:1905] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.872228 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 3298
I20260810 06:50:51.875948 4209 ts_tablet_manager.cc:1905] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:50:51.878422 4207 ts_tablet_manager.cc:1918] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:51.878562 4207 log.cc:1196] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/703dae1c194e4020a0cdb78e3f038263
I20260810 06:50:51.878995 4207 ts_tablet_manager.cc:1939] T 703dae1c194e4020a0cdb78e3f038263 P c1edcf5cefdf49fdaee772112fae5d54: Deleting consensus metadata
I20260810 06:50:51.880162 3138 catalog_manager.cc:4841] TS c1edcf5cefdf49fdaee772112fae5d54 (127.22.11.130:40647): tablet 703dae1c194e4020a0cdb78e3f038263 (table test-table [id=7dea448b9ea3403eae258e5e5d002d96]) successfully deleted
I20260810 06:50:51.881102 4210 ts_tablet_manager.cc:1918] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:51.881274 4210 log.cc:1196] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/703dae1c194e4020a0cdb78e3f038263
I20260810 06:50:51.881845 4210 ts_tablet_manager.cc:1939] T 703dae1c194e4020a0cdb78e3f038263 P f03f5db71d114bf4b2ec97c40ad7d62f: Deleting consensus metadata
I20260810 06:50:51.882551 4209 ts_tablet_manager.cc:1918] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 06:50:51.882705 4209 log.cc:1196] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/703dae1c194e4020a0cdb78e3f038263
I20260810 06:50:51.883162 4209 ts_tablet_manager.cc:1939] T 703dae1c194e4020a0cdb78e3f038263 P 98789c264f074fdf85c974e311e05fb5: Deleting consensus metadata
I20260810 06:50:51.884298 3136 catalog_manager.cc:4841] TS 98789c264f074fdf85c974e311e05fb5 (127.22.11.131:39037): tablet 703dae1c194e4020a0cdb78e3f038263 (table test-table [id=7dea448b9ea3403eae258e5e5d002d96]) successfully deleted
I20260810 06:50:51.884508 3132 catalog_manager.cc:4841] TS f03f5db71d114bf4b2ec97c40ad7d62f (127.22.11.129:36015): tablet 703dae1c194e4020a0cdb78e3f038263 (table test-table [id=7dea448b9ea3403eae258e5e5d002d96]) successfully deleted
Aug 10 06:50:51 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.129@KRBTEST.COM
I20260810 06:50:51.896234 4088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:51.874998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.22.11.129:36015)
0810 06:50:51.875160 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:51.875165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:51.880893 (+ 5728us) server_negotiation.cc:407] Connection header received
0810 06:50:51.881182 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:51.881188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:51.881302 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:51.881384 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:51.883146 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.884034 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.885494 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:51.885989 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:51.886098 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:51.890314 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:51.890327 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:51.890346 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:51.890380 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:51.893429 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:51.894247 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:51.894254 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:51.894256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:51.894378 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:51.894794 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:51.894797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:51.894799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:51.895182 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:51.895431 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:51.895755 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 06:50:51.895956 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:50:51.896893 3529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:52.047369 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 3298
I20260810 06:50:52.079312 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 3544
Aug 10 06:50:52 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.130@KRBTEST.COM
I20260810 06:50:52.093462 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:52.080109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46500 (local address 127.22.11.130:40647)
0810 06:50:52.080587 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:52.080592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:52.080609 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:50:52.080700 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:52.080704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:52.080766 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:52.080868 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:52.082112 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:52.082822 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:52.084112 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:52.084516 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:52.084735 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:52.088457 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:52.088471 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:52.088486 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:52.088521 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:52.091215 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:52.091759 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:52.091766 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:52.091768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:52.091896 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:52.092159 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:52.092161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:52.092163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:52.092493 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:52.092854 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:52.092983 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 06:50:52.093155 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":237,"threads_started":1}
I20260810 06:50:52.094030 3778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:52.223562 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 3544
I20260810 06:50:52.250586 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 3789
Aug 10 06:50:52 dist-test-slave-f7th krb5kdc[3085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.22.11.131@KRBTEST.COM
I20260810 06:50:52.264423 4260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:52.251299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34086 (local address 127.22.11.131:39037)
0810 06:50:52.251530 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:52.251534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:52.251552 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:50:52.251672 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:52.251676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:52.251738 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:52.251825 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:52.253079 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:52.253768 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:52.254977 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:52.255383 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:52.255510 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:52.259611 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:52.259626 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:52.259643 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:52.259677 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:52.262232 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:52.262791 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:52.262797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:52.262800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:52.262918 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:52.263186 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:52.263189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:52.263190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:52.263519 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:52.263764 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:52.263960 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 06:50:52.264130 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260810 06:50:52.265172 4017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:52.387409 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 3789
I20260810 06:50:52.415372 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 3105
I20260810 06:50:52.416505 3242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:52.528937 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 3105
2026-08-10T06:50:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6898 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:52 dist-test-slave-f7th krb5kdc[4281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:52 dist-test-slave-f7th krb5kdc[4281](info): set up 2 sockets
Aug 10 06:50:52 dist-test-slave-f7th krb5kdc[4281](info): commencing operation
krb5kdc: starting...
W20260810 06:50:53.776542 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:53 dist-test-slave-f7th krb5kdc[4281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:54.005772 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:38599
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:41563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:38599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:54.323861 4308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:54.324146 4308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:54.324282 4308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:54.331650 4308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:54.331758 4308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:54.331833 4308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:54.331908 4308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:54.331977 4308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:54.341619 4308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:41563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:38599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:38599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:54.342788 4308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:54.344040 4308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:54.355132 4315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:54.355093 4318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:54.355311 4314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:54.355921 4308 server_base.cc:1034] running on GCE node
I20260810 06:50:54.356637 4308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:54.357888 4308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:54.359076 4308 hybrid_clock.cc:648] HybridClock initialized: now 1786344654359025 us; error 42 us; skew 500 ppm
I20260810 06:50:54.359359 4308 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T06:50:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1897 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:50:54 dist-test-slave-f7th krb5kdc[4326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:50:54 dist-test-slave-f7th krb5kdc[4326](info): set up 2 sockets
Aug 10 06:50:54 dist-test-slave-f7th krb5kdc[4326](info): commencing operation
krb5kdc: starting...
W20260810 06:50:55.664700 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:50:55 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:50:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:50:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:50:55.898056 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40767
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:56.213133 4356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:56.213407 4356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:56.213505 4356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:56.220436 4356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:56.220533 4356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:56.220587 4356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:56.220679 4356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:56.220729 4356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:56.230047 4356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:56.231175 4356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:56.232369 4356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:56.243377 4367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:56.243320 4371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:56.243819 4366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:56.244222 4356 server_base.cc:1034] running on GCE node
I20260810 06:50:56.244989 4356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:56.246332 4356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:56.247536 4356 hybrid_clock.cc:648] HybridClock initialized: now 1786344656247480 us; error 42 us; skew 500 ppm
I20260810 06:50:56.247846 4356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:56 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344656, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:56.255263 4356 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:56.256651 4356 webserver.cc:460] Webserver started at http://127.22.11.190:42483/ using document root <none> and password file <none>
I20260810 06:50:56.257311 4356 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:56.257426 4356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:56.257714 4356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:56.260505 4356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "80725cac30c64499b3568e0187edcafa"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "88cdb8898a5b9e6070ffc7f21c3721b3"
server_key_iv: "eff7fa6f3a43868b4facf5a4bf9b240e"
server_key_version: "encryptionkey@0"
I20260810 06:50:56.261261 4356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "80725cac30c64499b3568e0187edcafa"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "88cdb8898a5b9e6070ffc7f21c3721b3"
server_key_iv: "eff7fa6f3a43868b4facf5a4bf9b240e"
server_key_version: "encryptionkey@0"
I20260810 06:50:56.266031 4356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260810 06:50:56.269202 4386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:56.270402 4356 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:50:56.270555 4356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "80725cac30c64499b3568e0187edcafa"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "88cdb8898a5b9e6070ffc7f21c3721b3"
server_key_iv: "eff7fa6f3a43868b4facf5a4bf9b240e"
server_key_version: "encryptionkey@0"
I20260810 06:50:56.270718 4356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:56.282590 4356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:56.286473 4356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:56.286731 4356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:56.303381 4356 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:40767
I20260810 06:50:56.303404 4466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:40767 every 8 connection(s)
I20260810 06:50:56.304680 4356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:56.306273 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 4356
I20260810 06:50:56.306493 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:50:56.307351 22574 external_mini_cluster.cc:1428] Setting key a2e792a3a071b44a5ad5edd8361d0b99
I20260810 06:50:56.308974 4468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:50:56.319705 4468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Bootstrap starting.
Aug 10 06:50:56 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:56.322778 4468 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Neither blocks nor log segments found. Creating new log.
I20260810 06:50:56.323791 4468 log.cc:824] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:56.326329 4468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: No bootstrap required, opened a new log
I20260810 06:50:56.327646 4473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:56.309380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.22.11.190:40767)
0810 06:50:56.310067 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:56.310078 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:56.310117 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:50:56.311661 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:56.311680 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:56.312029 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:56.312375 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:56.313846 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:56.315122 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:56.316342 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:56.317079 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:56.317258 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:56.321055 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:56.321063 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:56.321078 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:56.321116 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:56.324029 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:56.324672 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:56.324678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:56.324683 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:56.324844 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:56.325195 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:56.325198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:56.325200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:56.325673 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:56.325943 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:56.326360 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 06:50:56.326744 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":295,"threads_started":1}
I20260810 06:50:56.334111 4468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:56.334424 4468 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:50:56.334522 4468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80725cac30c64499b3568e0187edcafa, State: Initialized, Role: FOLLOWER
I20260810 06:50:56.335089 4468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:56.335342 4468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:56.335500 4468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:56.335676 4468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:50:56.337774 4468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:56.338300 4468 leader_election.cc:302] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80725cac30c64499b3568e0187edcafa; no voters:
I20260810 06:50:56.338951 4468 leader_election.cc:288] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:50:56.339063 4476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:50:56.339722 4476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 LEADER]: Becoming Leader. State: Replica: 80725cac30c64499b3568e0187edcafa, State: Running, Role: LEADER
I20260810 06:50:56.340284 4476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:56.340871 4468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:56.343241 4481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80725cac30c64499b3568e0187edcafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } } }
I20260810 06:50:56.343173 4482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80725cac30c64499b3568e0187edcafa. Latest consensus state: current_term: 1 leader_uuid: "80725cac30c64499b3568e0187edcafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } } }
I20260810 06:50:56.343606 4481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:56.343631 4482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:56.344442 4491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:56.350033 4491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:56.357946 4491 catalog_manager.cc:1347] Generated new cluster ID: d373a256fb68404f9e1dde93499acee5
I20260810 06:50:56.358045 4491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:56.375164 4491 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:50:56.376899 4491 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:56.395284 4491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Generated new TSK 0
I20260810 06:50:56.396031 4491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:50:56.454150 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40767
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:56.776131 4514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:56.776393 4514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:56.776494 4514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:56.783421 4514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:56.783522 4514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:56.783638 4514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:56.793147 4514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:56.794329 4514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:56.795610 4514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:56.808282 4524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:56.808355 4525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:56.808326 4528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:56.809119 4514 server_base.cc:1034] running on GCE node
I20260810 06:50:56.809670 4514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:56.810513 4514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:56.811703 4514 hybrid_clock.cc:648] HybridClock initialized: now 1786344656811657 us; error 38 us; skew 500 ppm
I20260810 06:50:56.811981 4514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:56 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344656, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:56.814944 4514 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:50:56.816283 4514 webserver.cc:460] Webserver started at http://127.22.11.129:35931/ using document root <none> and password file <none>
I20260810 06:50:56.816957 4514 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:56.817082 4514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:56.817360 4514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:56.820070 4514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "2aaa393d930249f9a9017a58e09c6913"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "069c454b7ecd6808db23e3a024195e46"
server_key_iv: "9a2e6e29caba63d2794feba2f1b1f2f4"
server_key_version: "encryptionkey@0"
I20260810 06:50:56.820863 4514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "2aaa393d930249f9a9017a58e09c6913"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "069c454b7ecd6808db23e3a024195e46"
server_key_iv: "9a2e6e29caba63d2794feba2f1b1f2f4"
server_key_version: "encryptionkey@0"
I20260810 06:50:56.825625 4514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:50:56.828836 4548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:56.830054 4514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:50:56.830209 4514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "2aaa393d930249f9a9017a58e09c6913"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "069c454b7ecd6808db23e3a024195e46"
server_key_iv: "9a2e6e29caba63d2794feba2f1b1f2f4"
server_key_version: "encryptionkey@0"
I20260810 06:50:56.830374 4514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:56.855067 4514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:56.859169 4514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:56.859428 4514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:56.860333 4514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:56.861640 4514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:56.861718 4514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:56.861838 4514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:56.861891 4514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:56.891116 4514 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:42179
I20260810 06:50:56.891175 4758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:42179 every 8 connection(s)
I20260810 06:50:56.892274 4514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:56.895807 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 4514
I20260810 06:50:56.896036 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:50:56.896857 22574 external_mini_cluster.cc:1428] Setting key 2cb66f6154e74222f109c98a0e33746c
Aug 10 06:50:56 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344656, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:56.912437 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:56.894581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:48315 (local address 127.22.11.190:40767)
0810 06:50:56.894973 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:56.894978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:56.896069 (+ 1091us) server_negotiation.cc:407] Connection header received
0810 06:50:56.898367 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:56.898372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:56.898441 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:56.898550 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:56.900396 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:56.901139 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:56.903076 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:56.903455 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:56.903549 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:56.907289 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:56.907294 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:56.907297 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:56.907329 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:56.909703 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:56.910326 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:56.910331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:56.910335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:56.910493 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:56.910849 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:56.910854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:56.910857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:56.911189 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:56.911336 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:56.911902 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 06:50:56.912120 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":190,"threads_started":1}
I20260810 06:50:56.913159 4769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:56.895073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:48315)
0810 06:50:56.895872 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:56.895937 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:56.898117 (+ 2180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:56.898768 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:56.898778 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:56.899490 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:56.900232 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:56.900257 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:56.901293 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:56.901297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:56.902947 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:56.902961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:56.903658 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:56.903664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:56.903916 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:56.904562 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:56.904589 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:56.907083 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:56.909829 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:56.909835 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:56.909845 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:56.910221 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:56.910582 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:56.910585 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:56.910587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:56.910750 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:56.911298 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:56.911303 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:56.911574 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 06:50:56.911784 (+ 210us) client_negotiation.cc:240] Negotiation successful
0810 06:50:56.912125 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":289,"threads_started":1}
I20260810 06:50:56.914511 4760 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40767
I20260810 06:50:56.914783 4760 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:56.915436 4760 heartbeater.cc:511] Master 127.22.11.190:40767 requested a full tablet report, sending...
I20260810 06:50:56.917459 4414 ts_manager.cc:194] Registered new tserver with Master: 2aaa393d930249f9a9017a58e09c6913 (127.22.11.129:42179)
I20260810 06:50:56.918830 4414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:48315
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:50:56.951078 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40767
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:57.270843 4772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:57.271147 4772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:57.271296 4772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:57.278434 4772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:57.278525 4772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:57.278669 4772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:57.287798 4772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:57.288982 4772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:57.290223 4772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:57.302435 4782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:57.302521 4777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:57.302475 4785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:57.303380 4772 server_base.cc:1034] running on GCE node
I20260810 06:50:57.304046 4772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:57.305155 4772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:57.306387 4772 hybrid_clock.cc:648] HybridClock initialized: now 1786344657306302 us; error 72 us; skew 500 ppm
I20260810 06:50:57.306728 4772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:57 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344657, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:57.310248 4772 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:50:57.313714 4772 webserver.cc:460] Webserver started at http://127.22.11.130:37199/ using document root <none> and password file <none>
I20260810 06:50:57.314339 4772 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:57.314440 4772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:57.314720 4772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:57.317553 4772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "b9cbfb5ad9e6404fa159607f1d05a3c3"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "6db5872de9e65aa8ba98d8ed8badfce5"
server_key_iv: "51b0f300dd91329d44ecfc04cefa78d1"
server_key_version: "encryptionkey@0"
I20260810 06:50:57.318275 4772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "b9cbfb5ad9e6404fa159607f1d05a3c3"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "6db5872de9e65aa8ba98d8ed8badfce5"
server_key_iv: "51b0f300dd91329d44ecfc04cefa78d1"
server_key_version: "encryptionkey@0"
I20260810 06:50:57.323081 4772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:50:57.326258 4801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:57.327494 4772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:50:57.327641 4772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "b9cbfb5ad9e6404fa159607f1d05a3c3"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "6db5872de9e65aa8ba98d8ed8badfce5"
server_key_iv: "51b0f300dd91329d44ecfc04cefa78d1"
server_key_version: "encryptionkey@0"
I20260810 06:50:57.327805 4772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:57.356457 4772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:57.360369 4772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:57.360648 4772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:57.361517 4772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:57.362774 4772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:57.362849 4772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:57.362967 4772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:57.363021 4772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:57.392218 4772 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35391
I20260810 06:50:57.392259 4996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35391 every 8 connection(s)
I20260810 06:50:57.393435 4772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:57.403143 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 4772
I20260810 06:50:57.403299 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:50:57.403841 22574 external_mini_cluster.cc:1428] Setting key 479fad07c3cc708290b2f2c7a187d6cf
Aug 10 06:50:57 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344657, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:57.412688 4763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:57.395918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:56943 (local address 127.22.11.190:40767)
0810 06:50:57.396084 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:57.396089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:57.397193 (+ 1104us) server_negotiation.cc:407] Connection header received
0810 06:50:57.399015 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:57.399020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:57.399100 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:57.399186 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:57.400711 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.401425 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.402919 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.403421 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.403533 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:57.407204 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:57.407209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:57.407213 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:57.407244 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:57.409779 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:57.410460 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:57.410464 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:57.410466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:57.410597 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:57.411009 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:57.411013 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:57.411015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:57.411262 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:57.411411 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:57.412079 (+ 668us) server_negotiation.cc:299] Negotiation successful
0810 06:50:57.412305 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:50:57.413602 5003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:57.396321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:56943)
0810 06:50:57.397037 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:57.397075 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:57.398747 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:57.399428 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:57.399437 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:57.399953 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:57.400496 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:57.400513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.401572 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.401576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:57.402775 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:57.402787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.403640 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.403655 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:57.403855 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:57.404492 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:57.404522 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:57.406989 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:57.409912 (+ 2923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:57.409920 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:57.409935 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:57.410332 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:57.410702 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:57.410707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:57.410711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:57.410897 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:57.411393 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:57.411400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:57.411740 (+ 340us) client_negotiation.cc:769] Sending connection context
0810 06:50:57.411974 (+ 234us) client_negotiation.cc:240] Negotiation successful
0810 06:50:57.412483 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":305,"threads_started":1}
I20260810 06:50:57.415005 5000 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40767
I20260810 06:50:57.415287 5000 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:57.415999 5000 heartbeater.cc:511] Master 127.22.11.190:40767 requested a full tablet report, sending...
I20260810 06:50:57.417498 4414 ts_manager.cc:194] Registered new tserver with Master: b9cbfb5ad9e6404fa159607f1d05a3c3 (127.22.11.130:35391)
I20260810 06:50:57.418303 4414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:56943
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:50:57.460979 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40767
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:57.787189 5010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:57.787452 5010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:57.787578 5010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:57.794580 5010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:57.794687 5010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:57.794806 5010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:50:57.804132 5010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:57.805416 5010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:57.806634 5010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:57.818677 5018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:57.818753 5017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:57.818714 5023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:57.819495 5010 server_base.cc:1034] running on GCE node
I20260810 06:50:57.820075 5010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:57.820966 5010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:57.822152 5010 hybrid_clock.cc:648] HybridClock initialized: now 1786344657822106 us; error 36 us; skew 500 ppm
I20260810 06:50:57.822511 5010 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:57 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344657, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:57.825579 5010 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:50:57.826871 5010 webserver.cc:460] Webserver started at http://127.22.11.131:35409/ using document root <none> and password file <none>
I20260810 06:50:57.827502 5010 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:57.827622 5010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:57.827906 5010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:50:57.830691 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "070efc7977a1459e966c0b398d8ef358"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "91445b0d73fe33c646d8e2c6c906de4e"
server_key_iv: "e0aa00080c964710073e5e695dc2f0bc"
server_key_version: "encryptionkey@0"
I20260810 06:50:57.831483 5010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "070efc7977a1459e966c0b398d8ef358"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "91445b0d73fe33c646d8e2c6c906de4e"
server_key_iv: "e0aa00080c964710073e5e695dc2f0bc"
server_key_version: "encryptionkey@0"
I20260810 06:50:57.836789 5010 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260810 06:50:57.840054 5033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:57.841465 5010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:57.841622 5010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "070efc7977a1459e966c0b398d8ef358"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "91445b0d73fe33c646d8e2c6c906de4e"
server_key_iv: "e0aa00080c964710073e5e695dc2f0bc"
server_key_version: "encryptionkey@0"
I20260810 06:50:57.841795 5010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:57.875128 5010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:57.879326 5010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:57.879604 5010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:57.880525 5010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:57.881836 5010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:57.881914 5010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:57.882045 5010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:57.882099 5010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:57.921476 4760 heartbeater.cc:503] Master 127.22.11.190:40767 was elected leader, sending a full tablet report...
I20260810 06:50:57.922901 5010 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:38533
I20260810 06:50:57.922914 5226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:38533 every 8 connection(s)
I20260810 06:50:57.924175 5010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:50:57.931458 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 5010
I20260810 06:50:57.931645 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:50:57.932282 22574 external_mini_cluster.cc:1428] Setting key bb6e712759d419ec6cf2c8ece32cf464
Aug 10 06:50:57 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344657, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:50:57.943727 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:57.926477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:46585 (local address 127.22.11.190:40767)
0810 06:50:57.926917 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:57.926921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:57.927933 (+ 1012us) server_negotiation.cc:407] Connection header received
0810 06:50:57.929761 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:57.929765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:57.929826 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:57.929908 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:57.931394 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.932544 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.934371 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.934856 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.934964 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:57.938257 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:57.938262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:57.938266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:57.938298 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:57.940816 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:57.941680 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:57.941686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:57.941689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:57.941844 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:57.942267 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:57.942270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:57.942272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:57.942557 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:57.942675 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:57.943230 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 06:50:57.943426 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":195,"threads_started":1}
I20260810 06:50:57.944519 5232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:57.927087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:46585)
0810 06:50:57.927741 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:57.927785 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:57.929507 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:57.930113 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:57.930120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:57.930609 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:57.931236 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:57.931253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.932717 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.932721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:57.934233 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:57.934247 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:57.935086 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:57.935094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:57.935302 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:57.935924 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:57.935950 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:50:57.938062 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:50:57.940941 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:57.940950 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:57.940966 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:57.941560 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:57.941962 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:50:57.941967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:50:57.941971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:50:57.942170 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:50:57.942653 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:50:57.942658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:50:57.942943 (+ 285us) client_negotiation.cc:769] Sending connection context
0810 06:50:57.943161 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 06:50:57.943511 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":268,"threads_started":1}
I20260810 06:50:57.945870 5227 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40767
I20260810 06:50:57.946195 5227 heartbeater.cc:464] Registering TS with master...
I20260810 06:50:57.947048 5227 heartbeater.cc:511] Master 127.22.11.190:40767 requested a full tablet report, sending...
I20260810 06:50:57.948401 4414 ts_manager.cc:194] Registered new tserver with Master: 070efc7977a1459e966c0b398d8ef358 (127.22.11.131:38533)
I20260810 06:50:57.949208 4414 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:46585
I20260810 06:50:57.959115 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab.
Entry for principal oryx/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab.
WARNING: no policy specified for oryx/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab.
Entry for principal oryx/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab.
WARNING: no policy specified for oryx/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab.
Entry for principal oryx/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab.
I20260810 06:50:58.107064 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 4514
Aug 10 06:50:58 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:50:58.122594 5250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:58.108250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.22.11.129:42179)
0810 06:50:58.108937 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:58.108943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:58.108969 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:50:58.109037 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:58.109044 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:58.109245 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:58.109404 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:58.110783 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:58.111821 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:58.113081 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:58.113669 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:58.113788 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:58.117658 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:58.117669 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:58.117692 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:58.117728 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:58.120233 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:58.120910 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:58.120917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:58.120920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:58.121056 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:58.121337 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:58.121340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:58.121342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:58.121677 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:58.121896 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:58.122102 (+ 206us) server_negotiation.cc:299] Negotiation successful
0810 06:50:58.122297 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":377,"threads_started":1}
I20260810 06:50:58.123837 4754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:58.204708 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 4514
I20260810 06:50:58.229926 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 4772
Aug 10 06:50:58 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:50:58.245357 5253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:58.230798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.22.11.130:35391)
0810 06:50:58.231304 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:58.231311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:58.231336 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:50:58.231397 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:58.231404 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:58.231611 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:58.231760 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:58.233083 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:58.234070 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:58.235291 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:58.235825 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:58.235936 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:58.239828 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:58.239838 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:58.239852 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:58.239892 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:58.242636 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:58.243257 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:58.243263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:58.243266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:58.243392 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:58.243746 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:58.243750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:58.243751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:58.244111 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:58.244397 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:58.244714 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 06:50:58.244974 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":247,"threads_started":1}
I20260810 06:50:58.246711 4986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:58.331027 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 4772
I20260810 06:50:58.350634 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5010
Aug 10 06:50:58 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:50:58.366199 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:58.351699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.22.11.131:38533)
0810 06:50:58.352209 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:58.352216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:58.352245 (+ 29us) server_negotiation.cc:407] Connection header received
0810 06:50:58.352309 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:58.352315 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:58.352526 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:58.352690 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:58.353899 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:58.354842 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:58.356048 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:58.356571 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:58.356802 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:58.361331 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:50:58.361342 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:50:58.361356 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:50:58.361388 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:50:58.363896 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:58.364452 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:58.364458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:58.364460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:58.364578 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:50:58.364944 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:50:58.364947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:50:58.364949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:50:58.365294 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:50:58.365509 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:50:58.365723 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:50:58.365909 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":247,"threads_started":1}
I20260810 06:50:58.367447 5221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:58.448264 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 5010
I20260810 06:50:58.474925 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 4356
I20260810 06:50:58.476050 4459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:50:58.577097 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 4356
I20260810 06:50:58.602655 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40767
--webserver_interface=127.22.11.190
--webserver_port=42483
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:50:58.929831 5258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:58.930145 5258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:58.930280 5258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:58.937836 5258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:50:58.937964 5258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:58.938037 5258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:58.938107 5258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:50:58.938166 5258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:50:58.948365 5258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=42483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:58.949746 5258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:58.951082 5258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:58.962467 5272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:58.962581 5273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:58.962834 5275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:58.963533 5258 server_base.cc:1034] running on GCE node
I20260810 06:50:58.964285 5258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:58.965696 5258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:58.966897 5258 hybrid_clock.cc:648] HybridClock initialized: now 1786344658966850 us; error 34 us; skew 500 ppm
I20260810 06:50:58.967233 5258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:58 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344658, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:58.970151 5258 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:50:58.971560 5258 webserver.cc:460] Webserver started at http://127.22.11.190:42483/ using document root <none> and password file <none>
I20260810 06:50:58.972206 5258 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:58.972337 5258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:58.977588 5258 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:50:58.980152 5285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:58.981405 5258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:50:58.981563 5258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "80725cac30c64499b3568e0187edcafa"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "88cdb8898a5b9e6070ffc7f21c3721b3"
server_key_iv: "eff7fa6f3a43868b4facf5a4bf9b240e"
server_key_version: "encryptionkey@0"
I20260810 06:50:58.982476 5258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:59.001250 5258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:59.004738 5258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:59.004967 5258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:59.021797 5258 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:40767
I20260810 06:50:59.021821 5384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:40767 every 8 connection(s)
I20260810 06:50:59.023200 5258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:50:59.026436 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 5258
I20260810 06:50:59.027218 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:42179
--local_ip_for_outbound_sockets=127.22.11.129
--tserver_master_addrs=127.22.11.190:40767
--webserver_port=35931
--webserver_interface=127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:50:59.029978 5386 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:50:59.033147 5386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Bootstrap starting.
I20260810 06:50:59.051455 5386 log.cc:824] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Log is configured to *not* fsync() on all Append() calls
I20260810 06:50:59.060235 5386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:50:59.060762 5386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Bootstrap complete.
I20260810 06:50:59.069392 5386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:59.070196 5386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80725cac30c64499b3568e0187edcafa, State: Initialized, Role: FOLLOWER
I20260810 06:50:59.070744 5386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:59.070958 5386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:50:59.071065 5386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:50:59.071223 5386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:50:59.074908 5386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:59.075374 5386 leader_election.cc:302] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80725cac30c64499b3568e0187edcafa; no voters:
I20260810 06:50:59.075923 5386 leader_election.cc:288] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:50:59.076110 5393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:50:59.077054 5393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [term 2 LEADER]: Becoming Leader. State: Replica: 80725cac30c64499b3568e0187edcafa, State: Running, Role: LEADER
I20260810 06:50:59.077602 5393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } }
I20260810 06:50:59.077907 5386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:50:59.080219 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80725cac30c64499b3568e0187edcafa. Latest consensus state: current_term: 2 leader_uuid: "80725cac30c64499b3568e0187edcafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } } }
I20260810 06:50:59.080858 5395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "80725cac30c64499b3568e0187edcafa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80725cac30c64499b3568e0187edcafa" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40767 } } }
I20260810 06:50:59.081154 5395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:59.080580 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa [sys.catalog]: This master's current role is: LEADER
I20260810 06:50:59.090046 5402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:50:59.096755 5402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:50:59.098100 5402 catalog_manager.cc:1259] Loaded cluster ID: d373a256fb68404f9e1dde93499acee5
I20260810 06:50:59.098203 5402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:50:59.102051 5402 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:50:59.103463 5402 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 80725cac30c64499b3568e0187edcafa: Loaded TSK: 0
I20260810 06:50:59.105020 5402 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:50:59.381124 5390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:59.381387 5390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:59.381506 5390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:59.388650 5390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:59.388752 5390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:59.388917 5390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:50:59.398378 5390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.129.keytab
--principal=oryx/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:42179
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=35931
--tserver_master_addrs=127.22.11.190:40767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:59.399616 5390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:59.400919 5390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:59.413931 5427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:59.413968 5428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:59.414913 5430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:59.416117 5390 server_base.cc:1034] running on GCE node
I20260810 06:50:59.416766 5390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:59.417690 5390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:59.418912 5390 hybrid_clock.cc:648] HybridClock initialized: now 1786344659418855 us; error 46 us; skew 500 ppm
I20260810 06:50:59.419200 5390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:59 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344659, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:59.422487 5390 init.cc:377] Logged in from keytab as oryx/127.22.11.129@KRBTEST.COM (short username oryx)
I20260810 06:50:59.423954 5390 webserver.cc:460] Webserver started at http://127.22.11.129:35931/ using document root <none> and password file <none>
I20260810 06:50:59.424664 5390 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:59.424798 5390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:59.430050 5390 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:50:59.432721 5438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:59.434062 5390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:50:59.434211 5390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "2aaa393d930249f9a9017a58e09c6913"
format_stamp: "Formatted at 2026-08-10 06:50:56 on dist-test-slave-f7th"
server_key: "069c454b7ecd6808db23e3a024195e46"
server_key_iv: "9a2e6e29caba63d2794feba2f1b1f2f4"
server_key_version: "encryptionkey@0"
I20260810 06:50:59.435115 5390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:59.464030 5390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:59.467758 5390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:59.468003 5390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:59.468971 5390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:59.470247 5390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:59.470341 5390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:59.470446 5390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:59.470515 5390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:59.500351 5390 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:42179
I20260810 06:50:59.500413 5627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:42179 every 8 connection(s)
I20260810 06:50:59.501741 5390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:50:59.503750 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 5390
I20260810 06:50:59.504513 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:35391
--local_ip_for_outbound_sockets=127.22.11.130
--tserver_master_addrs=127.22.11.190:40767
--webserver_port=37199
--webserver_interface=127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:50:59 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:50:59 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:50:59.836695 5635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:50:59.836999 5635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:50:59.837138 5635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:50:59.844265 5635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:50:59.844381 5635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:50:59.844538 5635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:50:59.853927 5635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.130.keytab
--principal=oryx/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:35391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=37199
--tserver_master_addrs=127.22.11.190:40767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:50:59.855175 5635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:50:59.856422 5635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:50:59.871243 5647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:59.871980 5646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:50:59.873221 5650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:50:59.873771 5635 server_base.cc:1034] running on GCE node
I20260810 06:50:59.874352 5635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:50:59.875267 5635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:50:59.876468 5635 hybrid_clock.cc:648] HybridClock initialized: now 1786344659876407 us; error 58 us; skew 500 ppm
I20260810 06:50:59.876786 5635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:50:59 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344659, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:50:59.879959 5635 init.cc:377] Logged in from keytab as oryx/127.22.11.130@KRBTEST.COM (short username oryx)
I20260810 06:50:59.881495 5635 webserver.cc:460] Webserver started at http://127.22.11.130:37199/ using document root <none> and password file <none>
I20260810 06:50:59.882125 5635 fs_manager.cc:359] Metadata directory not provided
I20260810 06:50:59.882247 5635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:50:59.887431 5635 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 06:50:59.889941 5663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:59.891209 5635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:50:59.891373 5635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "b9cbfb5ad9e6404fa159607f1d05a3c3"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "6db5872de9e65aa8ba98d8ed8badfce5"
server_key_iv: "51b0f300dd91329d44ecfc04cefa78d1"
server_key_version: "encryptionkey@0"
I20260810 06:50:59.892267 5635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:50:59.918642 5635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:50:59.922555 5635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:50:59.922813 5635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:50:59.923830 5635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:50:59.925283 5635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:50:59.925381 5635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:59.925482 5635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:50:59.925550 5635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:50:59.964740 5635 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35391
I20260810 06:50:59.964821 5856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35391 every 8 connection(s)
I20260810 06:50:59.966179 5635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:50:59.974188 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 5635
I20260810 06:50:59.975009 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:38533
--local_ip_for_outbound_sockets=127.22.11.131
--tserver_master_addrs=127.22.11.190:40767
--webserver_port=35409
--webserver_interface=127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:50:59 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:50:59 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 06:51:00.310482 5862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:00.310747 5862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:00.310909 5862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:00.318022 5862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:00.318128 5862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:00.318301 5862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:51:00.327549 5862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/krb5kdc/oryx_127.22.11.131.keytab
--principal=oryx/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:38533
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=35409
--tserver_master_addrs=127.22.11.190:40767
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:00.328835 5862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:00.330082 5862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:00.344939 5872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:00.344978 5875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:00.358592 5883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:00.358707 5862 server_base.cc:1034] running on GCE node
I20260810 06:51:00.359661 5862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:00.361073 5862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:00.362500 5862 hybrid_clock.cc:648] HybridClock initialized: now 1786344660362407 us; error 73 us; skew 500 ppm
I20260810 06:51:00.362947 5862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:51:00 dist-test-slave-f7th krb5kdc[4326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344660, etypes {rep=17 tkt=17 ses=17}, oryx/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:51:00.367722 5862 init.cc:377] Logged in from keytab as oryx/127.22.11.131@KRBTEST.COM (short username oryx)
I20260810 06:51:00.369916 5862 webserver.cc:460] Webserver started at http://127.22.11.131:35409/ using document root <none> and password file <none>
I20260810 06:51:00.370973 5862 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:00.371141 5862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:00.379410 5862 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:51:00.383255 5891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:00.385291 5862 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:51:00.385565 5862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "070efc7977a1459e966c0b398d8ef358"
format_stamp: "Formatted at 2026-08-10 06:50:57 on dist-test-slave-f7th"
server_key: "91445b0d73fe33c646d8e2c6c906de4e"
server_key_iv: "e0aa00080c964710073e5e695dc2f0bc"
server_key_version: "encryptionkey@0"
I20260810 06:51:00.387173 5862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:00.451328 5862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:00.457777 5862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:00.458194 5862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:00.459684 5862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:00.461779 5862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:00.461980 5862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:51:00.462145 5862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:00.462265 5862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:00.516268 5862 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:38533
I20260810 06:51:00.516378 6102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:38533 every 8 connection(s)
I20260810 06:51:00.518395 5862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:51:00.526664 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 5862
Aug 10 06:51:00 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:00 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:00.580780 6112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:00.545040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54954 (local address 127.22.11.190:40767)
0810 06:51:00.561004 (+ 15964us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:00.561008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:00.561023 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:51:00.561061 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:00.561065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:00.561123 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:00.561217 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:00.563864 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.564676 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.565922 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.566629 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.566745 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:00.570407 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:00.570420 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:00.570448 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:00.570506 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:00.574780 (+ 4274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:00.575760 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:00.575771 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:00.575783 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:00.576044 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:00.576523 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:00.576529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:00.576533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:00.577232 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:00.577583 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:00.579055 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0810 06:51:00.579575 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15855,"thread_start_us":15726,"threads_started":1}
W20260810 06:51:00.724645 5634 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.210s user 0.005s sys 0.011s
W20260810 06:51:00.726492 5633 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:59.504577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:58159 (local address 127.22.11.190:40767)
0810 06:50:59.505394 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:59.505411 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:59.505889 (+ 478us) server_negotiation.cc:407] Connection header received
0810 06:50:59.507972 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:59.507985 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:59.508300 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:59.508609 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:59.510096 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.511299 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.512835 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.513468 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.513642 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:00.726103 (+1212461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:58159: BlockingRecv error: recv got EOF from 127.22.11.129:58159 (error 108)
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":312,"threads_started":1}
I20260810 06:51:00.727553 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:59.504921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:58159)
0810 06:50:59.505720 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:59.505758 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:59.507414 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:59.508809 (+ 1395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:59.508821 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:59.509390 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:59.509913 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:59.509931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.511459 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.511463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:59.512708 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:59.512721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.513689 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.513697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:59.513946 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:59.514564 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:59.514593 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:00.725736 (+1211143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":369,"threads_started":1}
W20260810 06:51:00.728793 5629 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:40767 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.11.190:40767: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Aug 10 06:51:00 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:00 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:00.742252 5634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:00.729607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:40335)
0810 06:51:00.729844 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:00.729868 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:00.730006 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:00.730549 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:00.730554 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:00.731081 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:00.731323 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:00.731336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.732927 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.732935 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:00.734634 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:00.734653 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.735703 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.735710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:00.735905 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:00.736698 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:00.736723 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:00.741928 (+ 5205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260810 06:51:00.742479 5633 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:00.730036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:40335 (local address 127.22.11.190:40767)
0810 06:51:00.730220 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:00.730227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:00.730249 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:51:00.730311 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:00.730317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:00.730400 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:00.730513 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:00.731472 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.732755 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.734803 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.735511 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.735648 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:00.742201 (+ 6553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:40335: BlockingRecv error: recv got EOF from 127.22.11.129:40335 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 06:51:00 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:00 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:00.756182 6114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:00.743209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:45761)
0810 06:51:00.743906 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:00.743938 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:00.744138 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:00.744540 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:00.744545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:00.745540 (+ 995us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:00.745862 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:00.745876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.747495 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.747501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:00.749176 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:00.749191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.750237 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.750244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:00.750431 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:00.751169 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:00.751194 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:00.755828 (+ 4634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":721,"threads_started":1}
W20260810 06:51:00.756347 5633 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:00.743774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:45761 (local address 127.22.11.190:40767)
0810 06:51:00.743947 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:00.743953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:00.744028 (+ 75us) server_negotiation.cc:407] Connection header received
0810 06:51:00.744296 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:00.744302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:00.744399 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:00.744519 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:00.746084 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.747283 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.749332 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.750049 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.750183 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:00.756066 (+ 5883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:45761: BlockingRecv error: recv got EOF from 127.22.11.129:45761 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 06:51:00.756716 5629 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:51:01.117358 5861 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.139s user 0.007s sys 0.007s
W20260810 06:51:01.118661 5859 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:50:59.968942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:53907 (local address 127.22.11.190:40767)
0810 06:50:59.969387 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0810 06:50:59.969391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:50:59.970774 (+ 1383us) server_negotiation.cc:407] Connection header received
0810 06:50:59.972645 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:50:59.972652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:50:59.972744 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:50:59.972845 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:50:59.974351 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.975161 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.976756 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.977183 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.977302 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.118353 (+1141051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:53907: BlockingRecv error: recv got EOF from 127.22.11.130:53907 (error 108)
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":220,"threads_started":1}
I20260810 06:51:01.119329 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:50:59.969698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:53907)
0810 06:50:59.970594 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0810 06:50:59.970638 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:50:59.972337 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:50:59.973097 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:50:59.973105 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:50:59.973603 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:50:59.974177 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:59.974195 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.975311 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.975316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:59.976593 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:50:59.976606 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:50:59.977398 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:50:59.977405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:50:59.977632 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:50:59.978265 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:50:59.978295 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:01.118136 (+1139841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":362,"threads_started":1}
W20260810 06:51:01.120334 5857 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:40767 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.11.190:40767: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:01.130838 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:01.121025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:42099)
0810 06:51:01.121365 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:01.121380 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:01.121483 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:01.121811 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:01.121814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:01.122188 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:01.122367 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.122376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.123944 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.123949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.125008 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.125021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.125998 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.126003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.126130 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.126725 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:01.126744 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:01.130513 (+ 3769us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 06:51:01.130934 5859 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:01.121113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:42099 (local address 127.22.11.190:40767)
0810 06:51:01.121286 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:01.121291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:01.121445 (+ 154us) server_negotiation.cc:407] Connection header received
0810 06:51:01.121588 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:01.121594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:01.121686 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:01.121804 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:01.122494 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.123764 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.125131 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.125791 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.125926 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.130701 (+ 4775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:42099: BlockingRecv error: recv got EOF from 127.22.11.130:42099 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:01.140472 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:01.131712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:41561)
0810 06:51:01.131885 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:01.131906 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:01.132023 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:01.132737 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:01.132740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:01.133017 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:01.133161 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.133171 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.134641 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.134644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.135581 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.135592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.136484 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.136488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.136647 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.137162 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:01.137178 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:01.140281 (+ 3103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 06:51:01.140760 5857 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:51:01.140864 5859 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:01.132194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:41561 (local address 127.22.11.190:40767)
0810 06:51:01.132397 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:01.132403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:01.132422 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:51:01.132472 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:01.132478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:01.132568 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:01.132711 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:01.133290 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.134495 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.135700 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.136364 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.136487 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.140545 (+ 4058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:41561: BlockingRecv error: recv got EOF from 127.22.11.130:41561 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 06:51:01.570906 6105 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.013s user 0.007s sys 0.010s
W20260810 06:51:01.572186 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:00.522895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:51235 (local address 127.22.11.190:40767)
0810 06:51:00.545066 (+ 22171us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:00.545073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:00.548776 (+ 3703us) server_negotiation.cc:407] Connection header received
0810 06:51:00.548872 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:00.548877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:00.548970 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:00.549084 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:00.551545 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.552815 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.555168 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.555870 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.555962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.571973 (+1016011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:51235: BlockingRecv error: recv got EOF from 127.22.11.131:51235 (error 108)
Metrics: {"server-negotiator.queue_time_us":22034,"thread_start_us":21913,"threads_started":1}
I20260810 06:51:01.572798 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:00.522239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:51235)
0810 06:51:00.545699 (+ 23460us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:00.545757 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:00.548334 (+ 2577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:00.549649 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:00.549661 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:00.550479 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:00.551295 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:00.551323 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.553024 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.553030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:00.554992 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:00.555008 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:00.556333 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:00.556340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:00.556603 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:00.557427 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:00.557457 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:01.571792 (+1014335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23114,"thread_start_us":22848,"threads_started":1}
W20260810 06:51:01.573638 6103 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:40767 (0 consecutive failures): Not authorized: Failed to ping master at 127.22.11.190:40767: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:01.587903 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:01.574220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:57981)
0810 06:51:01.574417 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:01.574437 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:01.574551 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:01.574853 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:01.574857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:01.575258 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:01.575441 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.575451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.576456 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.576461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.577441 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.577454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.578239 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.578246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.578403 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.584297 (+ 5894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:01.584316 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:01.587639 (+ 3323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 06:51:01.588081 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:01.574301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:57981 (local address 127.22.11.190:40767)
0810 06:51:01.574464 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:01.574469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:01.574487 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:51:01.574661 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:01.574667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:01.574732 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:01.574813 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:01.575585 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.576314 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.577579 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.578065 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.578176 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.587851 (+ 9675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:57981: BlockingRecv error: recv got EOF from 127.22.11.131:57981 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:01.596275 6105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:01.588475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:38743)
0810 06:51:01.588650 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:01.588668 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:01.588766 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:01.589071 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:01.589074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:01.589345 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:01.589490 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.589509 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.590474 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.590477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.591471 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.591483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.592147 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.592150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.592274 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.592797 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:01.592812 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:01.596032 (+ 3220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:51:01.596524 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:01.588585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:38743 (local address 127.22.11.190:40767)
0810 06:51:01.588827 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:01.588831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:01.588845 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:51:01.588883 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:01.588909 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:01.588975 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:01.589048 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:01.589600 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.590359 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.591581 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.592036 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.592124 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.596239 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:38743: BlockingRecv error: recv got EOF from 127.22.11.131:38743 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
W20260810 06:51:01.596567 6103 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:01 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:01.765244 6115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:01.757264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:36329)
0810 06:51:01.757781 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:01.757798 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:01.757915 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:01.758184 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:01.758187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:01.758554 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:01.758706 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.758715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.759599 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.759602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.760528 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:01.760546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.761196 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.761200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:01.761313 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.761808 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:01.761824 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:01.765024 (+ 3200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":299,"threads_started":1}
W20260810 06:51:01.765388 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:01.757311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:36329 (local address 127.22.11.190:40767)
0810 06:51:01.757513 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:01.757520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:01.757873 (+ 353us) server_negotiation.cc:407] Connection header received
0810 06:51:01.758002 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:01.758005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:01.758082 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:01.758158 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:01.758809 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.759491 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.760666 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:01.761075 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:01.761161 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:01.765196 (+ 4035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:36329: BlockingRecv error: recv got EOF from 127.22.11.129:36329 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 06:51:02 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:02 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:02.150020 6116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:02.141309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:47289)
0810 06:51:02.141885 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:02.141906 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:02.142033 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:02.142348 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:02.142353 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:02.142803 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:02.142956 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:02.142965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.143997 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.144001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:02.144993 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:02.145006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.145728 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.145735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:02.145882 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:02.146429 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:02.146445 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:02.149703 (+ 3258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":328,"threads_started":1}
W20260810 06:51:02.150167 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:02.141407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:47289 (local address 127.22.11.190:40767)
0810 06:51:02.141600 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:02.141607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:02.141983 (+ 376us) server_negotiation.cc:407] Connection header received
0810 06:51:02.142146 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:02.142152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:02.142221 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:02.142301 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:02.143099 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.143850 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.145131 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.145557 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.145665 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:02.149922 (+ 4257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:47289: BlockingRecv error: recv got EOF from 127.22.11.130:47289 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 06:51:02 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:02 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:02.605525 6117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:02.597096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:42691)
0810 06:51:02.597625 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:02.597641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:02.597755 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:02.598039 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:02.598043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:02.598464 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:02.598617 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:02.598626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.599536 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.599540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:02.600461 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:02.600472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.601146 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.601150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:02.601266 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:02.601813 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:02.601830 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:02.605189 (+ 3359us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":281,"threads_started":1}
W20260810 06:51:02.605626 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:02.597162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:42691 (local address 127.22.11.190:40767)
0810 06:51:02.597314 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:02.597318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:02.597702 (+ 384us) server_negotiation.cc:407] Connection header received
0810 06:51:02.597840 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:02.597843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:02.597918 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:02.598004 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:02.598722 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.599426 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.600570 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.601017 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.601105 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:02.605409 (+ 4304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:42691: BlockingRecv error: recv got EOF from 127.22.11.131:42691 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 06:51:02 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:02 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:02.774787 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:02.766058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:60905)
0810 06:51:02.766592 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:02.766612 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:02.766760 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:02.767364 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:02.767368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:02.767861 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:02.768044 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:02.768053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.769217 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.769221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:02.770135 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:02.770146 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.770710 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.770714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:02.770829 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:02.771361 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:02.771378 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:02.774465 (+ 3087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":266,"threads_started":1}
W20260810 06:51:02.774966 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:02.766114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:60905 (local address 127.22.11.190:40767)
0810 06:51:02.766348 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:02.766355 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:02.766684 (+ 329us) server_negotiation.cc:407] Connection header received
0810 06:51:02.766890 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:02.766895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:02.766977 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:02.767076 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:02.768173 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.769097 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.770241 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:02.770613 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:02.770703 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:02.774723 (+ 4020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:60905: BlockingRecv error: recv got EOF from 127.22.11.129:60905 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 10 06:51:03 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:03 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:03.159582 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:03.150985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:38481)
0810 06:51:03.151472 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:03.151492 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:03.151630 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:03.151936 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:03.151941 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:03.152407 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:03.152597 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:03.152606 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.153584 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.153589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:03.154510 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:03.154522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.155269 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.155276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:03.155441 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:03.156026 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:03.156045 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:03.159333 (+ 3288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":274,"threads_started":1}
W20260810 06:51:03.159729 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:03.151091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:38481 (local address 127.22.11.190:40767)
0810 06:51:03.151276 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:03.151283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:03.151577 (+ 294us) server_negotiation.cc:407] Connection header received
0810 06:51:03.151741 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:03.151746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:03.151807 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:03.151890 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:03.152761 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.153453 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.154649 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.155120 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.155233 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:03.159528 (+ 4295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:38481: BlockingRecv error: recv got EOF from 127.22.11.130:38481 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:51:03 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:03 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:03.615043 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:03.606414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:54563)
0810 06:51:03.606904 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:03.606920 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:03.607054 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:03.607362 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:03.607367 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:03.607806 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:03.607995 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:03.608006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.609061 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.609067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:03.610001 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:03.610014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.610799 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.610804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:03.610981 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:03.611600 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:03.611624 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:03.614801 (+ 3177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":268,"threads_started":1}
W20260810 06:51:03.615187 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:03.606572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:54563 (local address 127.22.11.190:40767)
0810 06:51:03.606768 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:03.606774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:03.607011 (+ 237us) server_negotiation.cc:407] Connection header received
0810 06:51:03.607168 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:03.607172 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:03.607232 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:03.607310 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:03.608140 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.608931 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.610140 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.610652 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.610772 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:03.614975 (+ 4203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:54563: BlockingRecv error: recv got EOF from 127.22.11.131:54563 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 06:51:03 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:03 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:03.784201 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:03.775623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:35039)
0810 06:51:03.776105 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:03.776120 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:03.776252 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:03.776570 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:03.776574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:03.776998 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:03.777203 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:03.777213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.778271 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.778276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:03.779240 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:03.779253 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.779957 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.779963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:03.780152 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:03.780686 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:03.780702 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:03.783848 (+ 3146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":254,"threads_started":1}
W20260810 06:51:03.784271 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:03.775699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:35039 (local address 127.22.11.190:40767)
0810 06:51:03.775883 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:03.775890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:03.776197 (+ 307us) server_negotiation.cc:407] Connection header received
0810 06:51:03.776360 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:03.776366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:03.776436 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:03.776516 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:03.777332 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.778142 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.779394 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:03.779819 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:03.779927 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:03.784037 (+ 4110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:35039: BlockingRecv error: recv got EOF from 127.22.11.129:35039 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 06:51:04 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:04 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:04.168756 6125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:04.160401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:33285)
0810 06:51:04.160932 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:04.160952 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:04.161073 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:04.161344 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:04.161347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:04.161725 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:04.161896 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:04.161904 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.162815 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.162818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:04.163709 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:04.163721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.164316 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.164320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:04.164434 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:04.165013 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:04.165032 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:04.168502 (+ 3470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":288,"threads_started":1}
W20260810 06:51:04.168939 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:04.160489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:33285 (local address 127.22.11.190:40767)
0810 06:51:04.160665 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:04.160670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:04.161011 (+ 341us) server_negotiation.cc:407] Connection header received
0810 06:51:04.161177 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:04.161181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:04.161243 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:04.161313 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:04.162005 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.162710 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.163809 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.164216 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.164322 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:04.168715 (+ 4393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:33285: BlockingRecv error: recv got EOF from 127.22.11.130:33285 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 06:51:04 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:04 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:04.623968 6127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:04.615982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:55463)
0810 06:51:04.616502 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:04.616519 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:04.616659 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:04.616910 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:04.616913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:04.617286 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:04.617449 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:04.617457 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.618392 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.618395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:04.619290 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:04.619301 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.619927 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.619930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:04.620045 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:04.620560 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:04.620576 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:04.623716 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":271,"threads_started":1}
W20260810 06:51:04.624106 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:04.616036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:55463 (local address 127.22.11.190:40767)
0810 06:51:04.616201 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:04.616206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:04.616587 (+ 381us) server_negotiation.cc:407] Connection header received
0810 06:51:04.616744 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:04.616748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:04.616807 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:04.616882 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:04.617555 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.618284 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.619425 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.619819 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.619913 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:04.623903 (+ 3990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:55463: BlockingRecv error: recv got EOF from 127.22.11.131:55463 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 06:51:04 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:04 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:04.793943 6128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:04.785151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:54317)
0810 06:51:04.785703 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:04.785718 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:04.785830 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:04.786119 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:04.786122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:04.786531 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:04.786712 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:04.786722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.787670 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.787673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:04.788944 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:04.788959 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.789788 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.789794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:04.789914 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:04.790512 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:04.790557 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:04.793656 (+ 3099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":300,"threads_started":1}
W20260810 06:51:04.794025 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:04.785203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:54317 (local address 127.22.11.190:40767)
0810 06:51:04.785370 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:04.785375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:04.785776 (+ 401us) server_negotiation.cc:407] Connection header received
0810 06:51:04.785916 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:04.785919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:04.785979 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:04.786077 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:04.786826 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.787560 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.789069 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:04.789620 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:04.789741 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:04.793831 (+ 4090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:54317: BlockingRecv error: recv got EOF from 127.22.11.129:54317 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 06:51:05 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:05 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:05.177845 6135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:05.169793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:34969)
0810 06:51:05.170337 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:05.170364 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:05.170506 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:05.170779 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:05.170782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:05.171156 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:05.171322 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:05.171332 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.172215 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.172219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:05.173246 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:05.173260 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.173918 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.173922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:05.174042 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:05.174586 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:05.174603 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:05.177604 (+ 3001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":285,"threads_started":1}
W20260810 06:51:05.177953 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:05.169793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:34969 (local address 127.22.11.190:40767)
0810 06:51:05.169974 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:05.169981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:05.170433 (+ 452us) server_negotiation.cc:407] Connection header received
0810 06:51:05.170612 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:05.170616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:05.170678 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:05.170752 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:05.171423 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.172112 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.173362 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.173808 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.173898 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:05.177777 (+ 3879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:34969: BlockingRecv error: recv got EOF from 127.22.11.130:34969 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:51:05 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:05 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:05.633908 6136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:05.624871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:48303)
0810 06:51:05.625362 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:05.625378 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:05.625511 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:05.625828 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:05.625870 (+ 42us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:05.626290 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:05.626466 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:05.626476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.627506 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.627512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:05.628589 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:05.628606 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.629365 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.629370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:05.629562 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:05.630120 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:05.630137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:05.633566 (+ 3429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
W20260810 06:51:05.634090 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:05.624919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:48303 (local address 127.22.11.190:40767)
0810 06:51:05.625116 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:05.625123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:05.625462 (+ 339us) server_negotiation.cc:407] Connection header received
0810 06:51:05.625637 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:05.625641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:05.625704 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:05.625780 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:05.626611 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.627364 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.628736 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.629216 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.629348 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:05.633800 (+ 4452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:48303: BlockingRecv error: recv got EOF from 127.22.11.131:48303 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:51:05 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:05 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:05.803822 6138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:05.794982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:42397)
0810 06:51:05.795521 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:05.795541 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:05.795683 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:05.796002 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:05.796007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:05.796413 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:05.796569 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:05.796578 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.797546 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.797552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:05.798483 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:05.798494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.799168 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.799174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:05.799344 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:05.799990 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:05.800009 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:05.803601 (+ 3592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":262,"threads_started":1}
W20260810 06:51:05.804013 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:05.795076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:42397 (local address 127.22.11.190:40767)
0810 06:51:05.795279 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:05.795286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:05.795621 (+ 335us) server_negotiation.cc:407] Connection header received
0810 06:51:05.795793 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:05.795799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:05.795869 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:05.795948 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:05.796731 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.797420 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.798624 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:05.799021 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:05.799129 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:05.803783 (+ 4654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:42397: BlockingRecv error: recv got EOF from 127.22.11.129:42397 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:51:06 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:06 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:06.187577 6139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:06.178699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:41263)
0810 06:51:06.179220 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:06.179247 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:06.179408 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:06.179696 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:06.179700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:06.180113 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:06.180266 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:06.180275 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.181284 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.181288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:06.182210 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:06.182221 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.183023 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.183029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:06.183186 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:06.183778 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:06.183802 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:06.187349 (+ 3547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":296,"threads_started":1}
W20260810 06:51:06.187742 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:06.178863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:41263 (local address 127.22.11.190:40767)
0810 06:51:06.179018 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:06.179023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:06.179326 (+ 303us) server_negotiation.cc:407] Connection header received
0810 06:51:06.179512 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:06.179516 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:06.179584 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:06.179656 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:06.180384 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.181187 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.182352 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.182865 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.183000 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:06.187517 (+ 4517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:41263: BlockingRecv error: recv got EOF from 127.22.11.130:41263 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 06:51:06 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:06 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:06.647089 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:06.634932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:35359)
0810 06:51:06.635462 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:06.635479 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:06.635603 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:06.635854 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:06.635857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:06.636276 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:06.636430 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:06.636439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.637366 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.637370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:06.638358 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:06.638372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.638995 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.638999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:06.639146 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:06.643565 (+ 4419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:06.643584 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:06.646846 (+ 3262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":283,"threads_started":1}
W20260810 06:51:06.647251 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:06.635020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:35359 (local address 127.22.11.190:40767)
0810 06:51:06.635178 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:06.635182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:06.635543 (+ 361us) server_negotiation.cc:407] Connection header received
0810 06:51:06.635683 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:06.635687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:06.635747 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:06.635822 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:06.636544 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.637272 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.638485 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.638888 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.638984 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:06.647044 (+ 8060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:35359: BlockingRecv error: recv got EOF from 127.22.11.131:35359 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:51:06 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:06 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:06.812579 6144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:06.804703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:37733)
0810 06:51:06.805142 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:06.805158 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:06.805271 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:06.805526 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:06.805529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:06.805954 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:06.806134 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:06.806143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.807043 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.807047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:06.807935 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:06.807946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.808591 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.808595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:06.808741 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:06.809259 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:06.809276 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:06.812350 (+ 3074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":223,"threads_started":1}
W20260810 06:51:06.812830 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:06.804732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:37733 (local address 127.22.11.190:40767)
0810 06:51:06.804974 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:06.804980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:06.805229 (+ 249us) server_negotiation.cc:407] Connection header received
0810 06:51:06.805358 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:06.805361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:06.805423 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:06.805514 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:06.806237 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.806943 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.808034 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:06.808470 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:06.808570 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:06.812550 (+ 3980us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:37733: BlockingRecv error: recv got EOF from 127.22.11.129:37733 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:51:07 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:07 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:07.197074 6147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:07.188471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:55339)
0810 06:51:07.188954 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:07.188975 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:07.189098 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:07.189350 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:07.189353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:07.189747 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:07.189911 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:07.189920 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.190890 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.190894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:07.191878 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:07.191890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.192559 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.192564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:07.192738 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:07.193259 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:07.193275 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:07.196839 (+ 3564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":258,"threads_started":1}
W20260810 06:51:07.197211 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:07.188527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:55339 (local address 127.22.11.190:40767)
0810 06:51:07.188738 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:07.188746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:07.189043 (+ 297us) server_negotiation.cc:407] Connection header received
0810 06:51:07.189183 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:07.189187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:07.189248 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:07.189323 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:07.190031 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.190774 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.191983 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.192389 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.192491 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:07.197024 (+ 4533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:55339: BlockingRecv error: recv got EOF from 127.22.11.130:55339 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 06:51:07 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:07 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:07.656735 6150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:07.648003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:54839)
0810 06:51:07.648467 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:07.648482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:07.648658 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:07.648981 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:07.648986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:07.649412 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:07.649590 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:07.649599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.650603 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.650608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:07.651506 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:07.651519 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.652212 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.652218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:07.652372 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:07.652919 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:07.652936 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:07.656455 (+ 3519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":237,"threads_started":1}
W20260810 06:51:07.656883 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:07.648091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:54839 (local address 127.22.11.190:40767)
0810 06:51:07.648268 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:07.648275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:07.648557 (+ 282us) server_negotiation.cc:407] Connection header received
0810 06:51:07.648780 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:07.648785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:07.648851 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:07.648931 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:07.649728 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.650471 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.651651 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.652067 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.652172 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:07.656674 (+ 4502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:54839: BlockingRecv error: recv got EOF from 127.22.11.131:54839 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:51:07 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:07 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:07.822634 6152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:07.813511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:54617)
0810 06:51:07.814049 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:07.814065 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:07.814196 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:07.814521 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:07.814526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:07.814957 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:07.815279 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:07.815290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.816553 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.816557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:07.817540 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:07.817554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.818263 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.818269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:07.818426 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:07.818990 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:07.819009 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:07.822314 (+ 3305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":235,"threads_started":1}
W20260810 06:51:07.822749 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:07.813573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:54617 (local address 127.22.11.190:40767)
0810 06:51:07.813758 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:07.813765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:07.814145 (+ 380us) server_negotiation.cc:407] Connection header received
0810 06:51:07.814303 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:07.814308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:07.814369 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:07.814448 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:07.815448 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.816410 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.817689 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:07.818115 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:07.818225 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:07.822541 (+ 4316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:54617: BlockingRecv error: recv got EOF from 127.22.11.129:54617 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:51:08 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:08 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:08.206203 6153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:08.197898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:56979)
0810 06:51:08.198510 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:08.198532 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:08.198646 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:08.198908 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:08.198911 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:08.199311 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:08.199484 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:08.199494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.200404 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.200407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:08.201375 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:08.201386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.202033 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.202037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:08.202139 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:08.202702 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:08.202719 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:08.205852 (+ 3133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":269,"threads_started":1}
W20260810 06:51:08.206300 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:08.197965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:56979 (local address 127.22.11.190:40767)
0810 06:51:08.198142 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:08.198147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:08.198591 (+ 444us) server_negotiation.cc:407] Connection header received
0810 06:51:08.198725 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:08.198729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:08.198789 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:08.198864 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:08.199598 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.200305 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.201482 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.201919 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.202004 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:08.206066 (+ 4062us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:56979: BlockingRecv error: recv got EOF from 127.22.11.130:56979 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 06:51:08 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:08 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:08.666010 6154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:08.657732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:35623)
0810 06:51:08.658326 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:08.658342 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:08.658456 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:08.658715 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:08.658717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:08.659125 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:08.659309 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:08.659318 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.660243 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.660247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:08.661245 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:08.661258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.661888 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.661892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:08.662001 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:08.662586 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:08.662603 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:08.665779 (+ 3176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":266,"threads_started":1}
W20260810 06:51:08.666145 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:08.657810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:35623 (local address 127.22.11.190:40767)
0810 06:51:08.657975 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:08.657980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:08.658407 (+ 427us) server_negotiation.cc:407] Connection header received
0810 06:51:08.658542 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:08.658545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:08.658606 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:08.658682 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:08.659415 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.660130 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.661352 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.661774 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.661862 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:08.665947 (+ 4085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:35623: BlockingRecv error: recv got EOF from 127.22.11.131:35623 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 06:51:08 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:08 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:08.832082 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:08.823570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:50741)
0810 06:51:08.824101 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:08.824117 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:08.824230 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:08.824487 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:08.824491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:08.824909 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:08.825095 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:08.825104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.826074 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.826077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:08.827009 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:08.827021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.827703 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.827707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:08.827815 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:08.828409 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:08.828432 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:08.831739 (+ 3307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":268,"threads_started":1}
W20260810 06:51:08.832180 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:08.823749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:50741 (local address 127.22.11.190:40767)
0810 06:51:08.823978 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:08.823984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:08.824183 (+ 199us) server_negotiation.cc:407] Connection header received
0810 06:51:08.824317 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:08.824321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:08.824383 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:08.824460 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:08.825218 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.825958 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.827139 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:08.827565 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:08.827662 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:08.831941 (+ 4279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:50741: BlockingRecv error: recv got EOF from 127.22.11.129:50741 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 06:51:09 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:09 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:09.216079 6159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:09.207179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:56749)
0810 06:51:09.207683 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:09.207703 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:09.207815 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:09.208091 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:09.208094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:09.208488 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:09.208704 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:09.208716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.210248 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.210254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:09.211247 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:09.211260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.211978 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.211982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:09.212097 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:09.212682 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:09.212699 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:09.215844 (+ 3145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":271,"threads_started":1}
W20260810 06:51:09.216296 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:09.207248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:56749 (local address 127.22.11.190:40767)
0810 06:51:09.207490 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:09.207496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:09.207775 (+ 279us) server_negotiation.cc:407] Connection header received
0810 06:51:09.207913 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:09.207916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:09.207980 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:09.208060 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:09.208856 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.210091 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.211382 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.211806 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.211936 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:09.216068 (+ 4132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:56749: BlockingRecv error: recv got EOF from 127.22.11.130:56749 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 06:51:09 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:09 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:09.675290 6161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:09.666957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:51775)
0810 06:51:09.667441 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:09.667457 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:09.667567 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:09.667839 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:09.667842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:09.668262 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:09.668430 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:09.668438 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.669453 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.669457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:09.670388 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:09.670400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.671029 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.671033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:09.671150 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:09.671666 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:09.671682 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:09.675046 (+ 3364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":242,"threads_started":1}
W20260810 06:51:09.675453 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:09.667045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:51775 (local address 127.22.11.190:40767)
0810 06:51:09.667249 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:09.667256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:09.667537 (+ 281us) server_negotiation.cc:407] Connection header received
0810 06:51:09.667662 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:09.667667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:09.667730 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:09.667810 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:09.668557 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.669319 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.670489 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.670900 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.670988 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:09.675246 (+ 4258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:51775: BlockingRecv error: recv got EOF from 127.22.11.131:51775 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 06:51:09 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:09 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:09.841833 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:09.832930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:46197)
0810 06:51:09.833450 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:09.833466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:09.833604 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:09.833878 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:09.833881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:09.834294 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:09.834476 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:09.834490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.835800 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.835804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:09.836872 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:09.836885 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.837539 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.837544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:09.837671 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:09.838194 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:09.838209 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:09.841591 (+ 3382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":290,"threads_started":1}
W20260810 06:51:09.841981 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:09.832987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:46197 (local address 127.22.11.190:40767)
0810 06:51:09.833147 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:09.833152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:09.833557 (+ 405us) server_negotiation.cc:407] Connection header received
0810 06:51:09.833687 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:09.833691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:09.833769 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:09.833849 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:09.834655 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.835656 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.836989 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:09.837408 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:09.837502 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:09.841775 (+ 4273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:46197: BlockingRecv error: recv got EOF from 127.22.11.129:46197 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 10 06:51:10 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:10 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:10.225446 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:10.217006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:49363)
0810 06:51:10.217464 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:10.217484 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:10.217628 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:10.217951 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:10.217956 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:10.218374 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:10.218533 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:10.218542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.219540 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.219545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:10.220497 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:10.220510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.221229 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.221235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:10.221399 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:10.221908 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:10.221924 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:10.225203 (+ 3279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":223,"threads_started":1}
W20260810 06:51:10.225590 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:10.217034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:49363 (local address 127.22.11.190:40767)
0810 06:51:10.217208 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:10.217214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:10.217578 (+ 364us) server_negotiation.cc:407] Connection header received
0810 06:51:10.217737 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:10.217743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:10.217808 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:10.217887 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:10.218670 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.219408 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.220652 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.221087 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.221194 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:10.225367 (+ 4173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:49363: BlockingRecv error: recv got EOF from 127.22.11.130:49363 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 10 06:51:10 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:10 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:10.684598 6165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:10.676247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:42339)
0810 06:51:10.676778 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:10.676798 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:10.676933 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:10.677270 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:10.677273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:10.677662 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:10.677851 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:10.677862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.678842 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.678846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:10.679841 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:10.679854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.680536 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.680540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:10.680706 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:10.681213 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:10.681229 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:10.684354 (+ 3125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":246,"threads_started":1}
W20260810 06:51:10.684767 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:10.676260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:42339 (local address 127.22.11.190:40767)
0810 06:51:10.676474 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:10.676481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:10.676881 (+ 400us) server_negotiation.cc:407] Connection header received
0810 06:51:10.677053 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:10.677058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:10.677139 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:10.677223 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:10.677974 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.678726 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.679975 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.680412 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.680502 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:10.684522 (+ 4020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:42339: BlockingRecv error: recv got EOF from 127.22.11.131:42339 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 06:51:10 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:10 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:10.851539 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:10.842681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:47283)
0810 06:51:10.843171 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:10.843187 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:10.843333 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:10.843675 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:10.843680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:10.844121 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:10.844291 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:10.844301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.845355 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.845359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:10.846344 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:10.846357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.847104 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.847110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:10.847267 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:10.847829 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:10.847856 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:10.851245 (+ 3389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":240,"threads_started":1}
W20260810 06:51:10.851655 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:10.842826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:47283 (local address 127.22.11.190:40767)
0810 06:51:10.843052 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:10.843059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:10.843269 (+ 210us) server_negotiation.cc:407] Connection header received
0810 06:51:10.843456 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:10.843461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:10.843539 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:10.843624 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:10.844447 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.845208 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.846486 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:10.846946 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:10.847065 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:10.851444 (+ 4379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:47283: BlockingRecv error: recv got EOF from 127.22.11.129:47283 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 10 06:51:11 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:11 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:11.234694 6170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:11.226285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:50017)
0810 06:51:11.226774 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:11.226795 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:11.226923 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:11.227186 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:11.227189 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:11.227578 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:11.227739 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:11.227748 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.228889 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.228894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:11.230103 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:11.230117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.230746 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.230750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:11.230853 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:11.231425 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:11.231442 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:11.234458 (+ 3016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":249,"threads_started":1}
W20260810 06:51:11.234851 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:11.226355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:50017 (local address 127.22.11.190:40767)
0810 06:51:11.226515 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:11.226520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:11.226871 (+ 351us) server_negotiation.cc:407] Connection header received
0810 06:51:11.227010 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:11.227014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:11.227078 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:11.227155 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:11.227846 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.228738 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.230230 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.230634 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.230722 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:11.234636 (+ 3914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:50017: BlockingRecv error: recv got EOF from 127.22.11.130:50017 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 06:51:11 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:11 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:11.696785 6171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:11.685469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:49515)
0810 06:51:11.685988 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:11.686006 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:11.686116 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:11.686368 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:11.686371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:11.686742 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:11.686921 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:11.686931 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.687964 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.687969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:11.689099 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:11.689113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.689793 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.689798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:11.689907 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:11.693318 (+ 3411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:11.693336 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:11.696476 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":266,"threads_started":1}
W20260810 06:51:11.696867 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:11.685703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:49515 (local address 127.22.11.190:40767)
0810 06:51:11.685952 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:11.685958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:11.686068 (+ 110us) server_negotiation.cc:407] Connection header received
0810 06:51:11.686200 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:11.686203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:11.686264 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:11.686337 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:11.687048 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.687810 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.689232 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.689678 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.689770 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:11.696671 (+ 6901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:49515: BlockingRecv error: recv got EOF from 127.22.11.131:49515 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
Aug 10 06:51:11 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:11 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:11.860841 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:11.852374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:35617)
0810 06:51:11.852872 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:11.852888 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:11.853024 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:11.853356 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:11.853361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:11.853818 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:11.853999 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:11.854009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.855008 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.855013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:11.855931 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:11.855942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.856672 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.856679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:11.856829 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:11.857351 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:11.857367 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:11.860485 (+ 3118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":290,"threads_started":1}
W20260810 06:51:11.860955 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:11.852448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:35617 (local address 127.22.11.190:40767)
0810 06:51:11.852661 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:11.852667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:11.852969 (+ 302us) server_negotiation.cc:407] Connection header received
0810 06:51:11.853136 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:11.853142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:11.853222 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:11.853305 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:11.854142 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.854878 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.856070 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:11.856494 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:11.856604 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:11.860720 (+ 4116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:35617: BlockingRecv error: recv got EOF from 127.22.11.129:35617 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 06:51:12 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:12 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:12.244292 6177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:12.235492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:56709)
0810 06:51:12.235976 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:12.235996 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:12.236124 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:12.236429 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:12.236433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:12.236914 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:12.237100 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:12.237110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.238201 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.238204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:12.239195 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:12.239208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.239911 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.239917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:12.240104 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:12.240759 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:12.240784 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:12.244076 (+ 3292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":261,"threads_started":1}
W20260810 06:51:12.244448 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:12.235599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:56709 (local address 127.22.11.190:40767)
0810 06:51:12.235798 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:12.235804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:12.236076 (+ 272us) server_negotiation.cc:407] Connection header received
0810 06:51:12.236233 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:12.236239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:12.236306 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:12.236384 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:12.237240 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.238073 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.239336 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.239769 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.239875 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:12.244249 (+ 4374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:56709: BlockingRecv error: recv got EOF from 127.22.11.130:56709 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 06:51:12 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:12 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:12.706269 6180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:12.697625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:57003)
0810 06:51:12.698114 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:12.698129 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:12.698264 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:12.698572 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:12.698577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:12.699013 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:12.699200 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:12.699209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.700247 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.700252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:12.701287 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:12.701304 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.702163 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.702170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:12.702376 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:12.702930 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:12.702948 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:12.706055 (+ 3107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":285,"threads_started":1}
W20260810 06:51:12.706440 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:12.697714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:57003 (local address 127.22.11.190:40767)
0810 06:51:12.697906 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:12.697913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:12.698218 (+ 305us) server_negotiation.cc:407] Connection header received
0810 06:51:12.698380 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:12.698384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:12.698448 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:12.698528 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:12.699353 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.700115 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.701424 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.702003 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.702131 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:12.706225 (+ 4094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:57003: BlockingRecv error: recv got EOF from 127.22.11.131:57003 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 06:51:12 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:12 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:12.870618 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:12.861776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:53689)
0810 06:51:12.862351 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:12.862371 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:12.862499 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:12.862851 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:12.862856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:12.863274 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:12.863450 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:12.863459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.864696 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.864702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:12.865670 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:12.865682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.866469 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.866475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:12.866634 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:12.867231 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:12.867248 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:12.870386 (+ 3138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":283,"threads_started":1}
W20260810 06:51:12.870775 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:12.861839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:53689 (local address 127.22.11.190:40767)
0810 06:51:12.862055 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:12.862062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:12.862449 (+ 387us) server_negotiation.cc:407] Connection header received
0810 06:51:12.862627 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:12.862632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:12.862711 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:12.862792 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:12.863595 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.864516 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.865811 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:12.866281 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:12.866394 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:12.870561 (+ 4167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:53689: BlockingRecv error: recv got EOF from 127.22.11.129:53689 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 06:51:13 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:13 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:13.254580 6182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:13.245134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:47329)
0810 06:51:13.245687 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:13.245707 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:13.245829 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:13.246173 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:13.246178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:13.246591 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:13.246764 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:13.246774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.247771 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.247776 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:13.248737 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:13.248750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.249435 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.249442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:13.249606 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:13.250204 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:13.250226 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:13.254235 (+ 4009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":277,"threads_started":1}
W20260810 06:51:13.254741 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:13.245193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:47329 (local address 127.22.11.190:40767)
0810 06:51:13.245350 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:13.245354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:13.245799 (+ 445us) server_negotiation.cc:407] Connection header received
0810 06:51:13.245968 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:13.245972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:13.246037 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:13.246116 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:13.246917 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.247636 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.248869 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.249267 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.249373 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:13.254456 (+ 5083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:47329: BlockingRecv error: recv got EOF from 127.22.11.130:47329 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 06:51:13 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:13 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:13.716257 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:13.707086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:60947)
0810 06:51:13.707581 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:13.707596 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:13.707733 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:13.708041 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:13.708047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:13.708468 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:13.708674 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:13.708684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.709684 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.709689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:13.710612 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:13.710625 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.711432 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.711438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:13.711607 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:13.712243 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:13.712265 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:13.715965 (+ 3700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":241,"threads_started":1}
W20260810 06:51:13.716460 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:13.707136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:60947 (local address 127.22.11.190:40767)
0810 06:51:13.707311 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:13.707319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:13.707677 (+ 358us) server_negotiation.cc:407] Connection header received
0810 06:51:13.707846 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:13.707851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:13.707912 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:13.707992 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:13.708812 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.709554 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.710749 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.711247 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.711368 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:13.716210 (+ 4842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:60947: BlockingRecv error: recv got EOF from 127.22.11.131:60947 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:51:13 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:13 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:13.880714 6186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:13.871529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:55335)
0810 06:51:13.872143 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:13.872163 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:13.872306 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:13.872701 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:13.872706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:13.873144 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:13.873298 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:13.873307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.874269 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.874276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:13.875191 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:13.875204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.875927 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.875956 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:13.876162 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:13.876805 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:13.876830 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:13.880384 (+ 3554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":236,"threads_started":1}
W20260810 06:51:13.880854 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:13.871954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:55335 (local address 127.22.11.190:40767)
0810 06:51:13.872135 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:13.872142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:13.872245 (+ 103us) server_negotiation.cc:407] Connection header received
0810 06:51:13.872454 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:13.872460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:13.872552 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:13.872673 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:13.873435 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.874134 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.875334 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:13.875792 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:13.875903 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:13.880582 (+ 4679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:55335: BlockingRecv error: recv got EOF from 127.22.11.129:55335 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 10 06:51:14 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:14 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:14.264065 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:14.255621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:37345)
0810 06:51:14.256264 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:14.256285 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:14.256397 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:14.256704 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:14.256707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:14.257109 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:14.257254 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:14.257262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.258178 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.258181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:14.259070 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:14.259080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.259724 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.259728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:14.259844 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:14.260357 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:14.260373 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:14.263712 (+ 3339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":429,"threads_started":1}
W20260810 06:51:14.264147 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:14.255756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:37345 (local address 127.22.11.190:40767)
0810 06:51:14.255934 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:14.255940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:14.256360 (+ 420us) server_negotiation.cc:407] Connection header received
0810 06:51:14.256486 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:14.256490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:14.256564 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:14.256691 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:14.257360 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.258076 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.259173 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.259623 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.259711 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:14.263926 (+ 4215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:37345: BlockingRecv error: recv got EOF from 127.22.11.130:37345 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 06:51:14 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:14 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.131@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:14.725800 6189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:14.717177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.131:34125)
0810 06:51:14.717718 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:14.717737 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:14.717872 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:14.718174 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:14.718179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:14.718619 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:14.718767 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:14.718777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.719789 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.719795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:14.720763 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:14.720776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.721423 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.721428 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:14.721601 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:14.722103 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:14.722120 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:14.725481 (+ 3361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":292,"threads_started":1}
W20260810 06:51:14.726161 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:14.717540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:34125 (local address 127.22.11.190:40767)
0810 06:51:14.717720 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:14.717726 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:14.717794 (+ 68us) server_negotiation.cc:407] Connection header received
0810 06:51:14.717977 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:14.717983 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:14.718050 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:14.718128 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:14.718904 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.719658 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.720895 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.721290 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.721397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:14.725705 (+ 4308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.131:34125: BlockingRecv error: recv got EOF from 127.22.11.131:34125 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 06:51:14 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:14 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.129@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:14.890394 6190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:14.881799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.129:45415)
0810 06:51:14.882308 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:14.882324 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:14.882456 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:14.882790 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:14.882794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:14.883180 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:14.883367 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:14.883376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.884304 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.884308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:14.885351 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:14.885365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.886230 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.886236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:14.886432 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:14.887027 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:14.887045 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:14.890147 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":239,"threads_started":1}
W20260810 06:51:14.890524 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:14.881869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:45415 (local address 127.22.11.190:40767)
0810 06:51:14.882089 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:14.882093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:14.882394 (+ 301us) server_negotiation.cc:407] Connection header received
0810 06:51:14.882552 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:14.882555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:14.882628 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:14.882707 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:14.883479 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.884183 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.885494 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:14.886090 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:14.886208 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:14.890316 (+ 4108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.129:45415: BlockingRecv error: recv got EOF from 127.22.11.129:45415 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Aug 10 06:51:15 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 06:51:15 dist-test-slave-f7th krb5kdc[4326](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.22.11.130@KRBTEST.COM for oryx/127.22.11.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 06:51:15.273437 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:15.265353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40767 (local address 127.22.11.130:43883)
0810 06:51:15.265847 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:15.265871 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:15.266016 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:15.266341 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:15.266344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:15.266705 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:15.266908 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:15.266918 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.267795 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.267798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:15.268770 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:15.268782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.269360 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.269364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:15.269483 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:15.270020 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:15.270036 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:15.273174 (+ 3138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.22.11.190:40767: Server oryx/127.22.11.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":227,"threads_started":1}
W20260810 06:51:15.273567 6109 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:15.265429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:43883 (local address 127.22.11.190:40767)
0810 06:51:15.265628 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:15.265634 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:15.265957 (+ 323us) server_negotiation.cc:407] Connection header received
0810 06:51:15.266144 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:15.266149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:15.266227 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:15.266305 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:15.267015 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.267685 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.268872 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.269257 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.269350 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:15.273343 (+ 3993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.22.11.130:43883: BlockingRecv error: recv got EOF from 127.22.11.130:43883 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:51:15.528580 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5390
I20260810 06:51:15.540344 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:15.529609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52882 (local address 127.22.11.129:42179)
0810 06:51:15.530190 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:15.530197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:15.530222 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:51:15.530283 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:15.530289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:15.530530 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:15.530708 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:15.531996 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.533094 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.534245 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.534770 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.534868 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:15.537272 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:15.537281 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:15.537293 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:15.537333 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:15.539854 (+ 2521us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:51:15.540122 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52882: Request ticket server kudu/127.22.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":292,"threads_started":1}
W20260810 06:51:15.540448 6196 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52882: Request ticket server kudu/127.22.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:51:15.540454 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5390: Not authorized: Client connection negotiation failed: client connection to 127.22.11.129:42179: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.11.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:51:15.540689 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 5390
I20260810 06:51:15.565891 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5635
I20260810 06:51:15.577899 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:15.566986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57876 (local address 127.22.11.130:35391)
0810 06:51:15.567622 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:15.567630 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:15.567658 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:51:15.567723 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:15.567729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:15.567965 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:15.568126 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:15.569764 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.570838 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.571949 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.572509 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.572608 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:15.574896 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:15.574904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:15.574917 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:15.574954 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:15.577397 (+ 2443us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:51:15.577663 (+ 266us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57876: Request ticket server kudu/127.22.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":312,"threads_started":1}
W20260810 06:51:15.577961 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5635: Not authorized: Client connection negotiation failed: client connection to 127.22.11.130:35391: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:51:15.578006 6198 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57876: Request ticket server kudu/127.22.11.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:51:15.578097 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 5635
I20260810 06:51:15.603195 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5862
I20260810 06:51:15.615162 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:15.604271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36326 (local address 127.22.11.131:38533)
0810 06:51:15.604847 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:15.604854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:15.604878 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:51:15.604943 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:15.604950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:15.605199 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:15.605358 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:15.607078 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.608030 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.609306 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:15.609849 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:15.609946 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:15.612118 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:15.612126 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:15.612140 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:15.612178 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:15.614661 (+ 2483us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 06:51:15.614922 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36326: Request ticket server kudu/127.22.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":259,"threads_started":1}
W20260810 06:51:15.615204 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5862: Not authorized: Client connection negotiation failed: client connection to 127.22.11.131:38533: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.22.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 06:51:15.615252 6199 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36326: Request ticket server kudu/127.22.11.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 06:51:15.615346 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 5862
I20260810 06:51:15.640808 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 5258
I20260810 06:51:15.642212 5379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:15.758873 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 5258
2026-08-10T06:51:15Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21331 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T06:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:51:15Z Disabled control of system clock
I20260810 06:51:15.843832 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45841
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:45841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 06:51:16.168130 6204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:16.168395 6204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:16.168519 6204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:16.175566 6204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:51:16.175662 6204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:16.175725 6204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:16.175800 6204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:51:16.175858 6204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:51:16.184646 6204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:45841
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:16.185839 6204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:16.187642 6204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:16.198925 6215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:16.198951 6219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:16.198881 6216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:16.199730 6204 server_base.cc:1034] running on GCE node
I20260810 06:51:16.200430 6204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:16.201745 6204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:16.202934 6204 hybrid_clock.cc:648] HybridClock initialized: now 1786344676202896 us; error 30 us; skew 500 ppm
I20260810 06:51:16.203262 6204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:16.205335 6204 webserver.cc:460] Webserver started at http://127.22.11.190:34575/ using document root <none> and password file <none>
I20260810 06:51:16.205953 6204 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:16.206053 6204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:16.206362 6204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:16.209137 6204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "0460ccc1eb52465ca294e2184e5dec84"
format_stamp: "Formatted at 2026-08-10 06:51:16 on dist-test-slave-f7th"
server_key: "7f89c918a3495df9d7a293ec9e9bad18"
server_key_iv: "6a756ba4cc09c88db7979c10135943f7"
server_key_version: "encryptionkey@0"
I20260810 06:51:16.209867 6204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "0460ccc1eb52465ca294e2184e5dec84"
format_stamp: "Formatted at 2026-08-10 06:51:16 on dist-test-slave-f7th"
server_key: "7f89c918a3495df9d7a293ec9e9bad18"
server_key_iv: "6a756ba4cc09c88db7979c10135943f7"
server_key_version: "encryptionkey@0"
I20260810 06:51:16.214564 6204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:51:16.218055 6227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:16.219242 6204 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:51:16.219384 6204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "0460ccc1eb52465ca294e2184e5dec84"
format_stamp: "Formatted at 2026-08-10 06:51:16 on dist-test-slave-f7th"
server_key: "7f89c918a3495df9d7a293ec9e9bad18"
server_key_iv: "6a756ba4cc09c88db7979c10135943f7"
server_key_version: "encryptionkey@0"
I20260810 06:51:16.219568 6204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:16.236718 6204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:16.237476 6204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:16.237727 6204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:16.254652 6204 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:45841
I20260810 06:51:16.254670 6323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:45841 every 8 connection(s)
I20260810 06:51:16.255985 6204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:51:16.259791 6325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:51:16.265769 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 6204
I20260810 06:51:16.265944 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:51:16.266594 22574 external_mini_cluster.cc:1428] Setting key 55a3e332896377d3fd88b9c6b4b18732
I20260810 06:51:16.270864 6325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84: Bootstrap starting.
I20260810 06:51:16.274838 6325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84: Neither blocks nor log segments found. Creating new log.
I20260810 06:51:16.276034 6325 log.cc:824] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84: Log is configured to *not* fsync() on all Append() calls
I20260810 06:51:16.279151 6325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84: No bootstrap required, opened a new log
I20260810 06:51:16.282486 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:16.268727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.22.11.190:45841)
0810 06:51:16.269807 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:16.269822 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:16.269875 (+ 53us) server_negotiation.cc:407] Connection header received
0810 06:51:16.272149 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:16.272176 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:16.272818 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:16.273248 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:16.273780 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:16.275684 (+ 1904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:16.277428 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:16.278426 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:16.278617 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:16.279690 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:16.279699 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:16.279716 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:16.280394 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:16.280503 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:16.280535 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:16.280673 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:16.280893 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 06:51:16.281223 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":329,"threads_started":1}
I20260810 06:51:16.287443 6325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0460ccc1eb52465ca294e2184e5dec84" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45841 } }
I20260810 06:51:16.287758 6325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:51:16.287829 6325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0460ccc1eb52465ca294e2184e5dec84, State: Initialized, Role: FOLLOWER
I20260810 06:51:16.288290 6325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0460ccc1eb52465ca294e2184e5dec84" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45841 } }
I20260810 06:51:16.288476 6325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:51:16.288570 6325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:51:16.288722 6325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:51:16.290987 6325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0460ccc1eb52465ca294e2184e5dec84" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45841 } }
I20260810 06:51:16.291435 6325 leader_election.cc:302] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0460ccc1eb52465ca294e2184e5dec84; no voters:
I20260810 06:51:16.291958 6325 leader_election.cc:288] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:51:16.292059 6334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:51:16.292745 6334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [term 1 LEADER]: Becoming Leader. State: Replica: 0460ccc1eb52465ca294e2184e5dec84, State: Running, Role: LEADER
I20260810 06:51:16.293254 6334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0460ccc1eb52465ca294e2184e5dec84" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45841 } }
I20260810 06:51:16.294095 6325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:51:16.295928 6337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0460ccc1eb52465ca294e2184e5dec84. Latest consensus state: current_term: 1 leader_uuid: "0460ccc1eb52465ca294e2184e5dec84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0460ccc1eb52465ca294e2184e5dec84" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45841 } } }
I20260810 06:51:16.296412 6337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [sys.catalog]: This master's current role is: LEADER
I20260810 06:51:16.297022 6336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0460ccc1eb52465ca294e2184e5dec84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0460ccc1eb52465ca294e2184e5dec84" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45841 } } }
I20260810 06:51:16.297402 6336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84 [sys.catalog]: This master's current role is: LEADER
I20260810 06:51:16.301652 6341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:51:16.307325 6341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:51:16.315724 6341 catalog_manager.cc:1347] Generated new cluster ID: 986f6426d83847e7999305975fe4a5d8
I20260810 06:51:16.315827 6341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:51:16.352947 6341 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:51:16.354234 6341 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:51:16.364661 6341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0460ccc1eb52465ca294e2184e5dec84: Generated new TSK 0
I20260810 06:51:16.365469 6341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:51:16.380601 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45841
--builtin_ntp_servers=127.22.11.148:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 06:51:16.709846 6367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:16.710104 6367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:16.710232 6367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:16.717347 6367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:16.717446 6367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:16.717587 6367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:51:16.727456 6367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:16.728721 6367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:16.730597 6367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:16.743209 6376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:16.743110 6378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:16.743247 6386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:16.744400 6367 server_base.cc:1034] running on GCE node
I20260810 06:51:16.745064 6367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:16.745958 6367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:16.747155 6367 hybrid_clock.cc:648] HybridClock initialized: now 1786344676747107 us; error 38 us; skew 500 ppm
I20260810 06:51:16.747450 6367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:16.753620 6367 webserver.cc:460] Webserver started at http://127.22.11.129:41645/ using document root <none> and password file <none>
I20260810 06:51:16.754279 6367 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:16.754395 6367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:16.754703 6367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:16.757624 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "cb28e6021b4e48af876586f35da4eaba"
format_stamp: "Formatted at 2026-08-10 06:51:16 on dist-test-slave-f7th"
server_key: "1217e47f877302bd42e05ea3a45dd466"
server_key_iv: "ca0e75f7e7119c6f7aba688c7512aab2"
server_key_version: "encryptionkey@0"
I20260810 06:51:16.758380 6367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "cb28e6021b4e48af876586f35da4eaba"
format_stamp: "Formatted at 2026-08-10 06:51:16 on dist-test-slave-f7th"
server_key: "1217e47f877302bd42e05ea3a45dd466"
server_key_iv: "ca0e75f7e7119c6f7aba688c7512aab2"
server_key_version: "encryptionkey@0"
I20260810 06:51:16.763156 6367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:51:16.766342 6395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:16.767722 6367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:51:16.767897 6367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "cb28e6021b4e48af876586f35da4eaba"
format_stamp: "Formatted at 2026-08-10 06:51:16 on dist-test-slave-f7th"
server_key: "1217e47f877302bd42e05ea3a45dd466"
server_key_iv: "ca0e75f7e7119c6f7aba688c7512aab2"
server_key_version: "encryptionkey@0"
I20260810 06:51:16.768091 6367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:16.806152 6367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:16.807010 6367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:16.807305 6367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:16.808239 6367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:16.809696 6367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:16.809791 6367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:16.809902 6367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:16.809973 6367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:16.839788 6367 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:35953
I20260810 06:51:16.839848 6601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:35953 every 8 connection(s)
I20260810 06:51:16.841116 6367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:51:16.848062 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 6367
I20260810 06:51:16.848249 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:51:16.848951 22574 external_mini_cluster.cc:1428] Setting key 383dce55ad59289768ca74898e77fe4c
I20260810 06:51:16.851730 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45841
--builtin_ntp_servers=127.22.11.148:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:51:16.854202 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:16.843619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:43663 (local address 127.22.11.190:45841)
0810 06:51:16.844050 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:16.844054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:16.845295 (+ 1241us) server_negotiation.cc:407] Connection header received
0810 06:51:16.847366 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:16.847374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:16.847457 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:16.847558 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:16.848696 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:16.849515 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:16.851450 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:16.851847 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:16.851963 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:16.853399 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:16.853404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:16.853408 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:16.853429 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:16.853482 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:16.853499 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:16.853587 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:16.853783 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 06:51:16.853878 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":221,"threads_started":1}
I20260810 06:51:16.855088 6608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:16.844119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45841 (local address 127.22.11.129:43663)
0810 06:51:16.845123 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:16.845165 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:16.847053 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:16.847801 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:16.847809 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:16.847902 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:16.848501 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:16.848520 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:16.849679 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:16.849684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:16.851306 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:16.851322 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:16.852207 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:16.852213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:16.852417 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:16.853097 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:51:16.853099 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:16.853137 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:51:16.853219 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:16.853601 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:16.853610 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:16.853617 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:51:16.853757 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 06:51:16.854032 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":330,"threads_started":1}
I20260810 06:51:16.856534 6602 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45841
I20260810 06:51:16.856828 6602 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:16.857535 6602 heartbeater.cc:511] Master 127.22.11.190:45841 requested a full tablet report, sending...
I20260810 06:51:16.859668 6267 ts_manager.cc:194] Registered new tserver with Master: cb28e6021b4e48af876586f35da4eaba (127.22.11.129:35953)
I20260810 06:51:16.861560 6267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.129:43663
W20260810 06:51:17.176330 6609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:17.176605 6609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:17.176740 6609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:17.183696 6609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:17.183792 6609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:17.183940 6609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:51:17.192572 6609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:17.193743 6609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:17.195498 6609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:17.207908 6618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:17.207983 6619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:17.207950 6623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:17.209095 6609 server_base.cc:1034] running on GCE node
I20260810 06:51:17.209667 6609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:17.210553 6609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:17.211751 6609 hybrid_clock.cc:648] HybridClock initialized: now 1786344677211691 us; error 49 us; skew 500 ppm
I20260810 06:51:17.212041 6609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:17.214140 6609 webserver.cc:460] Webserver started at http://127.22.11.130:38861/ using document root <none> and password file <none>
I20260810 06:51:17.214797 6609 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:17.214897 6609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:17.215204 6609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:17.217952 6609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "771ab7581e73471c899bfa60c976c8cc"
format_stamp: "Formatted at 2026-08-10 06:51:17 on dist-test-slave-f7th"
server_key: "8bf310fd5dd07425e970e3543c18bdc5"
server_key_iv: "05a8a8791456a9657b1ff09f838b99f3"
server_key_version: "encryptionkey@0"
I20260810 06:51:17.218710 6609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "771ab7581e73471c899bfa60c976c8cc"
format_stamp: "Formatted at 2026-08-10 06:51:17 on dist-test-slave-f7th"
server_key: "8bf310fd5dd07425e970e3543c18bdc5"
server_key_iv: "05a8a8791456a9657b1ff09f838b99f3"
server_key_version: "encryptionkey@0"
I20260810 06:51:17.223480 6609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:51:17.226636 6633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:17.227906 6609 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 06:51:17.228056 6609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "771ab7581e73471c899bfa60c976c8cc"
format_stamp: "Formatted at 2026-08-10 06:51:17 on dist-test-slave-f7th"
server_key: "8bf310fd5dd07425e970e3543c18bdc5"
server_key_iv: "05a8a8791456a9657b1ff09f838b99f3"
server_key_version: "encryptionkey@0"
I20260810 06:51:17.228235 6609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:17.251024 6609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:17.251997 6609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:17.252313 6609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:17.253435 6609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:17.255054 6609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:17.255139 6609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:17.255237 6609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:17.255304 6609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:17.286510 6609 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:40757
I20260810 06:51:17.286581 6834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:40757 every 8 connection(s)
I20260810 06:51:17.287746 6609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:51:17.288206 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 6609
I20260810 06:51:17.288352 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:51:17.289048 22574 external_mini_cluster.cc:1428] Setting key a1d93ad777fa5e0fc35ac97e163297ef
I20260810 06:51:17.292292 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45841
--builtin_ntp_servers=127.22.11.148:38423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 06:51:17.300077 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:17.290552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:58789 (local address 127.22.11.190:45841)
0810 06:51:17.290743 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:17.290749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:17.292232 (+ 1483us) server_negotiation.cc:407] Connection header received
0810 06:51:17.294075 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:17.294081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:17.294156 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:17.294241 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:17.295341 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.296045 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.297572 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.297953 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.298072 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:17.299303 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:17.299308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:17.299311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:17.299327 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:17.299362 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:17.299374 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:17.299448 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:17.299652 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:51:17.299756 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:51:17.301023 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:17.291148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45841 (local address 127.22.11.130:58789)
0810 06:51:17.292085 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:17.292127 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:17.293823 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:17.294480 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:17.294488 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:17.294578 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:17.295183 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:17.295201 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.296194 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.296198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:17.297438 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:17.297450 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.298159 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.298167 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:17.298393 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:17.299015 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:51:17.299018 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:17.299054 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:51:17.299136 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:17.299460 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:17.299471 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:17.299478 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:51:17.299631 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 06:51:17.299993 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":309,"threads_started":1}
I20260810 06:51:17.302285 6838 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45841
I20260810 06:51:17.302584 6838 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:17.303328 6838 heartbeater.cc:511] Master 127.22.11.190:45841 requested a full tablet report, sending...
I20260810 06:51:17.304749 6267 ts_manager.cc:194] Registered new tserver with Master: 771ab7581e73471c899bfa60c976c8cc (127.22.11.130:40757)
I20260810 06:51:17.305527 6267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.130:58789
W20260810 06:51:17.619344 6842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:17.619585 6842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:17.619700 6842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:17.626780 6842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:17.626896 6842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:17.627043 6842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:51:17.635875 6842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:38423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:17.637106 6842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:17.638831 6842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:17.651084 6848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:17.651166 6856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:17.651129 6852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:17.652086 6842 server_base.cc:1034] running on GCE node
I20260810 06:51:17.652676 6842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:17.653570 6842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:17.654764 6842 hybrid_clock.cc:648] HybridClock initialized: now 1786344677654714 us; error 44 us; skew 500 ppm
I20260810 06:51:17.655072 6842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:17.657253 6842 webserver.cc:460] Webserver started at http://127.22.11.131:40993/ using document root <none> and password file <none>
I20260810 06:51:17.657897 6842 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:17.657997 6842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:17.658315 6842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:17.661132 6842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "a54b3bd2ed424f5da78f652687d30182"
format_stamp: "Formatted at 2026-08-10 06:51:17 on dist-test-slave-f7th"
server_key: "440140a5022746cec503e7b6c2558943"
server_key_iv: "6b67ff56da6997f873e32c0a7a645f89"
server_key_version: "encryptionkey@0"
I20260810 06:51:17.661880 6842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "a54b3bd2ed424f5da78f652687d30182"
format_stamp: "Formatted at 2026-08-10 06:51:17 on dist-test-slave-f7th"
server_key: "440140a5022746cec503e7b6c2558943"
server_key_iv: "6b67ff56da6997f873e32c0a7a645f89"
server_key_version: "encryptionkey@0"
I20260810 06:51:17.666627 6842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:51:17.669808 6866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:17.671000 6842 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:51:17.671152 6842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "a54b3bd2ed424f5da78f652687d30182"
format_stamp: "Formatted at 2026-08-10 06:51:17 on dist-test-slave-f7th"
server_key: "440140a5022746cec503e7b6c2558943"
server_key_iv: "6b67ff56da6997f873e32c0a7a645f89"
server_key_version: "encryptionkey@0"
I20260810 06:51:17.671337 6842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:17.684979 6842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:17.685734 6842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:17.686008 6842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:17.686865 6842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:17.688269 6842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:17.688365 6842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:17.688479 6842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:17.688549 6842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:17.718616 6842 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:46265
I20260810 06:51:17.718665 7075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:46265 every 8 connection(s)
I20260810 06:51:17.719825 6842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:51:17.732074 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 6842
I20260810 06:51:17.732252 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:51:17.732831 22574 external_mini_cluster.cc:1428] Setting key 6e2b6a8f280d6ce4ef29cd9ce87fa369
I20260810 06:51:17.734895 6606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:17.722197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:40277 (local address 127.22.11.190:45841)
0810 06:51:17.722357 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:17.722361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:17.723681 (+ 1320us) server_negotiation.cc:407] Connection header received
0810 06:51:17.726093 (+ 2412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:17.726100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:17.726187 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:17.726257 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:17.727906 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.729066 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.731393 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.732023 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.732119 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:17.734006 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:17.734011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:17.734015 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:17.734033 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:17.734068 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:17.734080 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:17.734158 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:17.734457 (+ 299us) server_negotiation.cc:299] Negotiation successful
0810 06:51:17.734564 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:51:17.736047 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:17.722658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45841 (local address 127.22.11.131:40277)
0810 06:51:17.723529 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:17.723571 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:17.725752 (+ 2181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:17.726775 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:17.726785 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:17.726906 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:17.727696 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:17.727718 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.729265 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.729270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:17.731237 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:17.731252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.732515 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.732523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:17.732854 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:17.733632 (+ 778us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:51:17.733636 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:17.733692 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:51:17.733785 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:17.734171 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:17.734183 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:17.734192 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 06:51:17.734433 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 06:51:17.734815 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":296,"threads_started":1}
I20260810 06:51:17.737346 7076 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45841
I20260810 06:51:17.737646 7076 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:17.738363 7076 heartbeater.cc:511] Master 127.22.11.190:45841 requested a full tablet report, sending...
I20260810 06:51:17.739647 6267 ts_manager.cc:194] Registered new tserver with Master: a54b3bd2ed424f5da78f652687d30182 (127.22.11.131:46265)
I20260810 06:51:17.740389 6267 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.131:40277
I20260810 06:51:17.748299 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:51:17.754040 6606 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:17.752487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.22.11.190:45841)
0810 06:51:17.752695 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:17.752700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:17.753149 (+ 449us) server_negotiation.cc:407] Connection header received
0810 06:51:17.753296 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:17.753302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:17.753375 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:17.753456 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:17.753862 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53208: BlockingRecv error: recv got EOF from 127.0.0.1:53208 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:51:17.754765 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 6367
I20260810 06:51:17.761998 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:17.755594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34346 (local address 127.22.11.129:35953)
0810 06:51:17.756077 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:17.756084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:17.756108 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:51:17.756160 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:17.756166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:17.756346 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:17.756463 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:17.756852 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.757812 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.759028 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.759575 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.759685 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:17.760604 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:17.760629 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:17.760646 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:17.761218 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:17.761275 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:17.761295 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:17.761396 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:17.761548 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 06:51:17.761689 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":244,"threads_started":1}
I20260810 06:51:17.763090 6591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:17.843573 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 6367
I20260810 06:51:17.868346 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 6609
I20260810 06:51:17.875795 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:17.869275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60008 (local address 127.22.11.130:40757)
0810 06:51:17.869773 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:17.869780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:17.869808 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:51:17.869872 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:17.869879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:17.870104 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:17.870228 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:17.870579 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.871570 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.872834 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.873379 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.873472 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:17.874429 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:17.874437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:17.874450 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:17.875026 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:17.875081 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:17.875101 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:17.875184 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:17.875293 (+ 109us) server_negotiation.cc:299] Negotiation successful
0810 06:51:17.875447 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":258,"threads_started":1}
I20260810 06:51:17.877058 6830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:17.960561 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 6609
I20260810 06:51:17.986951 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 6842
I20260810 06:51:17.995136 7097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:17.987925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.22.11.131:46265)
0810 06:51:17.988605 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:17.988611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:17.988749 (+ 138us) server_negotiation.cc:407] Connection header received
0810 06:51:17.988817 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:17.988824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:17.989041 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:17.989191 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:17.989650 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.990772 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.991974 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:17.992528 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:17.992664 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:17.993695 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:17.993703 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:17.993717 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:17.994283 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:17.994360 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:17.994396 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:17.994505 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:17.994643 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 06:51:17.994803 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":320,"threads_started":1}
I20260810 06:51:17.996482 7069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:18.076958 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 6842
I20260810 06:51:18.102461 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 6204
I20260810 06:51:18.103669 6319 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:18.204757 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 6204
2026-08-10T06:51:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2447 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T06:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:51:18Z Disabled control of system clock
I20260810 06:51:18.290900 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45949
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:45949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:51:18.609578 7105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:18.609840 7105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:18.609965 7105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:18.616899 7105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:51:18.616994 7105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:18.617066 7105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:18.617146 7105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:51:18.617213 7105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:51:18.625916 7105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:45949
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:18.627331 7105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:18.629141 7105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:18.640547 7114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:18.640591 7115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:18.640638 7121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:18.641258 7105 server_base.cc:1034] running on GCE node
I20260810 06:51:18.641973 7105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:18.643277 7105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:18.644474 7105 hybrid_clock.cc:648] HybridClock initialized: now 1786344678644437 us; error 29 us; skew 500 ppm
I20260810 06:51:18.644833 7105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:18.646947 7105 webserver.cc:460] Webserver started at http://127.22.11.190:42265/ using document root <none> and password file <none>
I20260810 06:51:18.647594 7105 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:18.647680 7105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:18.647950 7105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:18.650840 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd"
format_stamp: "Formatted at 2026-08-10 06:51:18 on dist-test-slave-f7th"
server_key: "decce36ea2559802d29caa1b9b48e5ab"
server_key_iv: "854523f33f60a38fb6ddda05ac20f2b8"
server_key_version: "encryptionkey@0"
I20260810 06:51:18.651587 7105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd"
format_stamp: "Formatted at 2026-08-10 06:51:18 on dist-test-slave-f7th"
server_key: "decce36ea2559802d29caa1b9b48e5ab"
server_key_iv: "854523f33f60a38fb6ddda05ac20f2b8"
server_key_version: "encryptionkey@0"
I20260810 06:51:18.656373 7105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260810 06:51:18.659531 7129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:18.660787 7105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:51:18.660920 7105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd"
format_stamp: "Formatted at 2026-08-10 06:51:18 on dist-test-slave-f7th"
server_key: "decce36ea2559802d29caa1b9b48e5ab"
server_key_iv: "854523f33f60a38fb6ddda05ac20f2b8"
server_key_version: "encryptionkey@0"
I20260810 06:51:18.661105 7105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:18.666299 7105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:18.666978 7105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:18.667227 7105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:18.682546 7105 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:45949
I20260810 06:51:18.682585 7227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:45949 every 8 connection(s)
I20260810 06:51:18.683713 7105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:51:18.687194 7228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:51:18.692309 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 7105
I20260810 06:51:18.692503 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:51:18.693135 22574 external_mini_cluster.cc:1428] Setting key f4e6c944887fb228f8b68031b162cf81
I20260810 06:51:18.697577 7228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd: Bootstrap starting.
I20260810 06:51:18.700879 7228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd: Neither blocks nor log segments found. Creating new log.
I20260810 06:51:18.701100 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:18.695137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.22.11.190:45949)
0810 06:51:18.696114 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:18.696123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:18.696164 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:51:18.697811 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:18.697832 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:18.698195 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:18.698480 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:18.699565 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:18.699571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:18.699583 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:18.699597 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:18.699651 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:18.699669 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:18.699836 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:18.699964 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 06:51:18.700196 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":355,"threads_started":1}
I20260810 06:51:18.701960 7228 log.cc:824] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:51:18.704921 7228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd: No bootstrap required, opened a new log
I20260810 06:51:18.711983 7228 raft_consensus.cc:348] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45949 } }
I20260810 06:51:18.712308 7228 raft_consensus.cc:374] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:51:18.712407 7228 raft_consensus.cc:729] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f544e5e6cb7d4e7a8d4dfba04ffdf5fd, State: Initialized, Role: FOLLOWER
I20260810 06:51:18.713016 7228 consensus_queue.cc:260] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45949 } }
I20260810 06:51:18.713284 7228 raft_consensus.cc:388] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:51:18.713393 7228 raft_consensus.cc:482] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:51:18.713510 7228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:51:18.715471 7228 raft_consensus.cc:504] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45949 } }
I20260810 06:51:18.715879 7228 leader_election.cc:302] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f544e5e6cb7d4e7a8d4dfba04ffdf5fd; no voters:
I20260810 06:51:18.716378 7228 leader_election.cc:288] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:51:18.716472 7236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:51:18.717135 7236 raft_consensus.cc:686] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [term 1 LEADER]: Becoming Leader. State: Replica: f544e5e6cb7d4e7a8d4dfba04ffdf5fd, State: Running, Role: LEADER
I20260810 06:51:18.717675 7236 consensus_queue.cc:237] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45949 } }
I20260810 06:51:18.718199 7228 sys_catalog.cc:564] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:51:18.720831 7237 sys_catalog.cc:455] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45949 } } }
I20260810 06:51:18.720918 7239 sys_catalog.cc:455] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f544e5e6cb7d4e7a8d4dfba04ffdf5fd. Latest consensus state: current_term: 1 leader_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f544e5e6cb7d4e7a8d4dfba04ffdf5fd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45949 } } }
I20260810 06:51:18.721230 7237 sys_catalog.cc:458] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [sys.catalog]: This master's current role is: LEADER
I20260810 06:51:18.721263 7239 sys_catalog.cc:458] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd [sys.catalog]: This master's current role is: LEADER
I20260810 06:51:18.722558 7251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:51:18.727353 7251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:51:18.735401 7251 catalog_manager.cc:1347] Generated new cluster ID: 61f55ddddcc447e5aea66885ef79d5a3
I20260810 06:51:18.735502 7251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:51:18.763345 7251 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:51:18.763859 7251 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:51:18.777115 7251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f544e5e6cb7d4e7a8d4dfba04ffdf5fd: Generated new TSK 0
I20260810 06:51:18.778090 7251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:51:18.783985 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45949
--builtin_ntp_servers=127.22.11.148:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 06:51:19.113700 7267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:19.113957 7267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:19.114085 7267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:19.121186 7267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:19.121281 7267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:19.121433 7267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:51:19.130219 7267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:19.131369 7267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:19.133112 7267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:19.145337 7286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:19.145380 7277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:19.145460 7278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:19.146023 7267 server_base.cc:1034] running on GCE node
I20260810 06:51:19.146559 7267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:19.147413 7267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:19.148633 7267 hybrid_clock.cc:648] HybridClock initialized: now 1786344679148566 us; error 40 us; skew 500 ppm
I20260810 06:51:19.148973 7267 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:19.155735 7267 webserver.cc:460] Webserver started at http://127.22.11.129:41897/ using document root <none> and password file <none>
I20260810 06:51:19.156420 7267 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:19.156515 7267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:19.156865 7267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:19.162166 7267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "65416d2558ec4853bee82ac3dd950096"
format_stamp: "Formatted at 2026-08-10 06:51:19 on dist-test-slave-f7th"
server_key: "b5e2c469da8af7f3f01a2b31faf2bb5e"
server_key_iv: "3e1bb302440318b894832a900eef3a41"
server_key_version: "encryptionkey@0"
I20260810 06:51:19.162922 7267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "65416d2558ec4853bee82ac3dd950096"
format_stamp: "Formatted at 2026-08-10 06:51:19 on dist-test-slave-f7th"
server_key: "b5e2c469da8af7f3f01a2b31faf2bb5e"
server_key_iv: "3e1bb302440318b894832a900eef3a41"
server_key_version: "encryptionkey@0"
I20260810 06:51:19.167940 7267 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:51:19.171200 7293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:19.172391 7267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:51:19.172526 7267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "65416d2558ec4853bee82ac3dd950096"
format_stamp: "Formatted at 2026-08-10 06:51:19 on dist-test-slave-f7th"
server_key: "b5e2c469da8af7f3f01a2b31faf2bb5e"
server_key_iv: "3e1bb302440318b894832a900eef3a41"
server_key_version: "encryptionkey@0"
I20260810 06:51:19.172761 7267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:19.178131 7267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:19.178861 7267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:19.179111 7267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:19.179961 7267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:19.181356 7267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:19.181444 7267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:19.181550 7267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:19.181624 7267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:19.211637 7267 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:42007
I20260810 06:51:19.211705 7506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:42007 every 8 connection(s)
I20260810 06:51:19.212970 7267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:51:19.220829 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:19.215418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:39205 (local address 127.22.11.190:45949)
0810 06:51:19.215850 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:19.215854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:19.217042 (+ 1188us) server_negotiation.cc:407] Connection header received
0810 06:51:19.218723 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:19.218728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:19.218791 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:19.218910 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:19.220100 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:19.220106 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:19.220110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:19.220128 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:19.220168 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:19.220180 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:19.220290 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:19.220464 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:51:19.220528 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":217,"threads_started":1}
I20260810 06:51:19.221272 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 7267
I20260810 06:51:19.221491 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:51:19.221783 7511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:19.215991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45949 (local address 127.22.11.129:39205)
0810 06:51:19.216897 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:19.216932 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:19.218463 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:19.219144 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:19.219152 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:19.219238 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:19.219822 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:51:19.219824 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:19.219859 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:51:19.219940 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:19.220265 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:19.220277 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:19.220285 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:51:19.220552 (+ 267us) client_negotiation.cc:240] Negotiation successful
0810 06:51:19.220814 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":251,"threads_started":1}
I20260810 06:51:19.222226 22574 external_mini_cluster.cc:1428] Setting key 9fc8ee43f0a0ddd9da30011bd0d89174
I20260810 06:51:19.223498 7507 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45949
I20260810 06:51:19.223759 7507 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:19.224413 7507 heartbeater.cc:511] Master 127.22.11.190:45949 requested a full tablet report, sending...
I20260810 06:51:19.225222 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45949
--builtin_ntp_servers=127.22.11.148:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 06:51:19.226447 7157 ts_manager.cc:194] Registered new tserver with Master: 65416d2558ec4853bee82ac3dd950096 (127.22.11.129:42007)
W20260810 06:51:19.565428 7514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:19.565673 7514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:19.565766 7514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:19.573140 7514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:19.573228 7514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:19.573339 7514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:51:19.581879 7514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:19.582965 7514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:19.584638 7514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:19.596902 7523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:19.596911 7522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:19.596891 7526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:19.597820 7514 server_base.cc:1034] running on GCE node
I20260810 06:51:19.598382 7514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:19.599221 7514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:19.600407 7514 hybrid_clock.cc:648] HybridClock initialized: now 1786344679600352 us; error 50 us; skew 500 ppm
I20260810 06:51:19.600725 7514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:19.602764 7514 webserver.cc:460] Webserver started at http://127.22.11.130:40691/ using document root <none> and password file <none>
I20260810 06:51:19.603399 7514 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:19.603497 7514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:19.603803 7514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:19.606535 7514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "b7e6310f125c4577849690d9359f5bc0"
format_stamp: "Formatted at 2026-08-10 06:51:19 on dist-test-slave-f7th"
server_key: "396ff8cae032d9662558913a52caf841"
server_key_iv: "a40946dba1fa26171101fdaa766dab5b"
server_key_version: "encryptionkey@0"
I20260810 06:51:19.607292 7514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "b7e6310f125c4577849690d9359f5bc0"
format_stamp: "Formatted at 2026-08-10 06:51:19 on dist-test-slave-f7th"
server_key: "396ff8cae032d9662558913a52caf841"
server_key_iv: "a40946dba1fa26171101fdaa766dab5b"
server_key_version: "encryptionkey@0"
I20260810 06:51:19.612006 7514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:51:19.615139 7541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:19.616286 7514 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:51:19.616425 7514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "b7e6310f125c4577849690d9359f5bc0"
format_stamp: "Formatted at 2026-08-10 06:51:19 on dist-test-slave-f7th"
server_key: "396ff8cae032d9662558913a52caf841"
server_key_iv: "a40946dba1fa26171101fdaa766dab5b"
server_key_version: "encryptionkey@0"
I20260810 06:51:19.616637 7514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:19.621846 7514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:19.622558 7514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:19.622793 7514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:19.623657 7514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:19.624981 7514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:19.625090 7514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:19.625193 7514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:19.625262 7514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:19.655336 7514 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:34843
I20260810 06:51:19.655386 7736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:34843 every 8 connection(s)
I20260810 06:51:19.656648 7514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:51:19.660385 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 7514
I20260810 06:51:19.660593 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:51:19.661051 22574 external_mini_cluster.cc:1428] Setting key 1345d2e0ca18f34c0f72bb1078e0d26b
I20260810 06:51:19.663827 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45949
--builtin_ntp_servers=127.22.11.148:45081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 06:51:19.664876 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:19.659059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:58815 (local address 127.22.11.190:45949)
0810 06:51:19.659237 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:19.659241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:19.660651 (+ 1410us) server_negotiation.cc:407] Connection header received
0810 06:51:19.662570 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:19.662574 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:19.662623 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:19.662723 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:19.664038 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:19.664044 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:19.664049 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:19.664068 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:19.664103 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:19.664115 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:19.664195 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:19.664498 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 06:51:19.664565 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:51:19.665783 7741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:19.659568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45949 (local address 127.22.11.130:58815)
0810 06:51:19.660462 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:19.660535 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:19.662276 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:19.662976 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:19.662987 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:19.663098 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:19.663710 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:51:19.663713 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:19.663749 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:51:19.663832 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:19.664297 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:19.664304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:19.664309 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:51:19.664568 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 06:51:19.664836 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":266,"threads_started":1}
I20260810 06:51:19.667040 7737 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45949
I20260810 06:51:19.667294 7737 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:19.667829 7737 heartbeater.cc:511] Master 127.22.11.190:45949 requested a full tablet report, sending...
I20260810 06:51:19.669157 7157 ts_manager.cc:194] Registered new tserver with Master: b7e6310f125c4577849690d9359f5bc0 (127.22.11.130:34843)
W20260810 06:51:19.988219 7742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:19.988492 7742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:19.988627 7742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:19.996272 7742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:19.996369 7742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:19.996511 7742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:51:20.005326 7742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:45949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:20.006503 7742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:20.008250 7742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:20.020648 7751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:20.020449 7755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:20.020674 7760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:20.021654 7742 server_base.cc:1034] running on GCE node
I20260810 06:51:20.022214 7742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:20.023051 7742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:20.024204 7742 hybrid_clock.cc:648] HybridClock initialized: now 1786344680024153 us; error 40 us; skew 500 ppm
I20260810 06:51:20.024549 7742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:51:20.026638 7742 webserver.cc:460] Webserver started at http://127.22.11.131:38583/ using document root <none> and password file <none>
I20260810 06:51:20.027266 7742 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:20.027355 7742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:20.027719 7742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:20.030390 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "3e3587c10f1340a9b4711fa12ea4b972"
format_stamp: "Formatted at 2026-08-10 06:51:20 on dist-test-slave-f7th"
server_key: "c8c34cefa91848b539ae143713bc23dc"
server_key_iv: "59bbd6bf82a0c2596ac82794e0963ff1"
server_key_version: "encryptionkey@0"
I20260810 06:51:20.031136 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "3e3587c10f1340a9b4711fa12ea4b972"
format_stamp: "Formatted at 2026-08-10 06:51:20 on dist-test-slave-f7th"
server_key: "c8c34cefa91848b539ae143713bc23dc"
server_key_iv: "59bbd6bf82a0c2596ac82794e0963ff1"
server_key_version: "encryptionkey@0"
I20260810 06:51:20.035840 7742 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260810 06:51:20.039073 7768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:20.040242 7742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:51:20.040381 7742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "3e3587c10f1340a9b4711fa12ea4b972"
format_stamp: "Formatted at 2026-08-10 06:51:20 on dist-test-slave-f7th"
server_key: "c8c34cefa91848b539ae143713bc23dc"
server_key_iv: "59bbd6bf82a0c2596ac82794e0963ff1"
server_key_version: "encryptionkey@0"
I20260810 06:51:20.040572 7742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:20.045680 7742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:20.046373 7742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:20.046613 7742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:20.047446 7742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:20.048792 7742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:20.048882 7742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:20.048985 7742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:20.049059 7742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:20.078365 7742 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:35837
I20260810 06:51:20.078402 7984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:35837 every 8 connection(s)
I20260810 06:51:20.079592 7742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:51:20.087252 7510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:20.081959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:53151 (local address 127.22.11.190:45949)
0810 06:51:20.082137 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:20.082142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:20.083450 (+ 1308us) server_negotiation.cc:407] Connection header received
0810 06:51:20.085172 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:20.085178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:20.085240 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:20.085326 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:20.086539 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:20.086545 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:20.086550 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:20.086568 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:20.086599 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:20.086611 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:20.086711 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:20.086903 (+ 192us) server_negotiation.cc:299] Negotiation successful
0810 06:51:20.086970 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:51:20.088055 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:20.082440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:45949 (local address 127.22.11.131:53151)
0810 06:51:20.083306 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:20.083347 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:20.084921 (+ 1574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:20.085554 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:20.085562 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:20.085645 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:20.086256 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:51:20.086258 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:20.086293 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:51:20.086375 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:20.086696 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:20.086706 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:20.086714 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:51:20.086968 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 06:51:20.087196 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":265,"threads_started":1}
I20260810 06:51:20.088797 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 7742
I20260810 06:51:20.088970 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:51:20.089294 7988 heartbeater.cc:347] Connected to a master server at 127.22.11.190:45949
I20260810 06:51:20.089462 22574 external_mini_cluster.cc:1428] Setting key e2e966c58332629f13843e1d399609f6
I20260810 06:51:20.089594 7988 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:20.090220 7988 heartbeater.cc:511] Master 127.22.11.190:45949 requested a full tablet report, sending...
I20260810 06:51:20.091465 7157 ts_manager.cc:194] Registered new tserver with Master: 3e3587c10f1340a9b4711fa12ea4b972 (127.22.11.131:35837)
I20260810 06:51:20.092371 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 06:51:20.098129 7510 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:51:20.096749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.22.11.190:45949)
0810 06:51:20.096889 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:20.096893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:20.097422 (+ 529us) server_negotiation.cc:407] Connection header received
0810 06:51:20.097538 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:20.097542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:20.097589 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:20.097661 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:20.097897 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55308: BlockingRecv error: recv got EOF from 127.0.0.1:55308 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:51:20.098589 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 7267
I20260810 06:51:20.101518 8006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:20.099471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.22.11.129:42007)
0810 06:51:20.099915 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:20.099921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:20.099944 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:51:20.099998 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:20.100003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:20.100160 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:20.100270 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:20.101018 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:20.101023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:20.101032 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:20.101043 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:20.101080 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:20.101095 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:20.101194 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:20.101235 (+ 41us) server_negotiation.cc:299] Negotiation successful
0810 06:51:20.101309 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":205,"threads_started":1}
I20260810 06:51:20.102532 7496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:20.181224 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 7267
I20260810 06:51:20.205852 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 7514
I20260810 06:51:20.209147 8008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:20.206752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33052 (local address 127.22.11.130:34843)
0810 06:51:20.207266 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:20.207274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:20.207301 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:51:20.207365 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:20.207370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:20.207583 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:20.207713 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:20.208480 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:20.208486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:20.208498 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:20.208510 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:20.208555 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:20.208573 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:20.208718 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:20.208822 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 06:51:20.208915 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":232,"threads_started":1}
I20260810 06:51:20.210337 7732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:20.289222 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 7514
I20260810 06:51:20.310238 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 7742
I20260810 06:51:20.313853 8011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:20.311129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38640 (local address 127.22.11.131:35837)
0810 06:51:20.311816 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:20.311822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:20.311854 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:51:20.311916 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:20.311922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:20.312149 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:20.312274 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:20.313197 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:20.313204 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:20.313215 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:51:20.313231 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:20.313270 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:51:20.313283 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:20.313408 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:20.313460 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 06:51:20.313549 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":358,"threads_started":1}
I20260810 06:51:20.315061 7976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:20.394429 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 7742
I20260810 06:51:20.419457 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 7105
I20260810 06:51:20.420660 7222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:20.521360 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 7105
2026-08-10T06:51:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2315 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:51:20 dist-test-slave-f7th krb5kdc[8023](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:51:20 dist-test-slave-f7th krb5kdc[8023](info): set up 2 sockets
Aug 10 06:51:20 dist-test-slave-f7th krb5kdc[8023](info): commencing operation
krb5kdc: starting...
W20260810 06:51:21.777174 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:51:21 dist-test-slave-f7th krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:51:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:51:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:51:22.027928 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40365
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:51:22.347373 8047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:22.347626 8047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:22.347738 8047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:22.354570 8047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:51:22.354660 8047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:22.354734 8047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:22.354815 8047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:51:22.354871 8047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:51:22.364010 8047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:22.365262 8047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:22.366468 8047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:22.377384 8059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:22.377372 8063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:22.377485 8060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:22.378038 8047 server_base.cc:1034] running on GCE node
I20260810 06:51:22.378763 8047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:22.380036 8047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:22.381222 8047 hybrid_clock.cc:648] HybridClock initialized: now 1786344682381181 us; error 34 us; skew 500 ppm
I20260810 06:51:22.381539 8047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:51:22 dist-test-slave-f7th krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344682, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:51:22.386971 8047 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:51:22.388275 8047 webserver.cc:460] Webserver started at http://127.22.11.190:39789/ using document root <none> and password file <none>
I20260810 06:51:22.388948 8047 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:22.389068 8047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:22.389343 8047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:22.392127 8047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "1aef36fe61d34b74acbac958b9bf8a1b"
format_stamp: "Formatted at 2026-08-10 06:51:22 on dist-test-slave-f7th"
server_key: "bf474d96d31f563c902662eb818d0fde"
server_key_iv: "884bdb08a86f196c93ba9db1e0593f0e"
server_key_version: "encryptionkey@0"
I20260810 06:51:22.392881 8047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "1aef36fe61d34b74acbac958b9bf8a1b"
format_stamp: "Formatted at 2026-08-10 06:51:22 on dist-test-slave-f7th"
server_key: "bf474d96d31f563c902662eb818d0fde"
server_key_iv: "884bdb08a86f196c93ba9db1e0593f0e"
server_key_version: "encryptionkey@0"
I20260810 06:51:22.397617 8047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:51:22.400758 8076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:22.401938 8047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:51:22.402097 8047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "1aef36fe61d34b74acbac958b9bf8a1b"
format_stamp: "Formatted at 2026-08-10 06:51:22 on dist-test-slave-f7th"
server_key: "bf474d96d31f563c902662eb818d0fde"
server_key_iv: "884bdb08a86f196c93ba9db1e0593f0e"
server_key_version: "encryptionkey@0"
I20260810 06:51:22.402261 8047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:22.416821 8047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:22.420874 8047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:22.421147 8047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:22.438314 8047 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:40365
I20260810 06:51:22.438344 8171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:40365 every 8 connection(s)
I20260810 06:51:22.439592 8047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:51:22.440302 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 8047
I20260810 06:51:22.440533 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:51:22.441212 22574 external_mini_cluster.cc:1428] Setting key 956d67bcf9357c16ba0c48c1aba725f4
I20260810 06:51:22.443830 8173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:51:22 dist-test-slave-f7th krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:51:22.456005 8173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b: Bootstrap starting.
I20260810 06:51:22.459892 8173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b: Neither blocks nor log segments found. Creating new log.
I20260810 06:51:22.461107 8173 log.cc:824] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b: Log is configured to *not* fsync() on all Append() calls
I20260810 06:51:22.464300 8173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b: No bootstrap required, opened a new log
I20260810 06:51:22.464407 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:22.443073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.22.11.190:40365)
0810 06:51:22.443931 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:22.443947 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:22.443999 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:51:22.446241 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:22.446265 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:22.446724 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:22.447169 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:22.448428 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:22.450478 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:22.451666 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:22.452664 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:22.452841 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:22.456363 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:22.456373 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:22.456391 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:22.456440 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:22.460134 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:22.460797 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:22.460805 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:22.460812 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:22.461036 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:22.461406 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:22.461411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:22.461414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:22.462041 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:22.462282 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:22.462738 (+ 456us) server_negotiation.cc:299] Negotiation successful
0810 06:51:22.463152 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":325,"threads_started":1}
I20260810 06:51:22.472267 8173 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40365 } }
I20260810 06:51:22.472733 8173 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:51:22.472846 8173 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aef36fe61d34b74acbac958b9bf8a1b, State: Initialized, Role: FOLLOWER
I20260810 06:51:22.473344 8173 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40365 } }
I20260810 06:51:22.473549 8173 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:51:22.473642 8173 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:51:22.473765 8173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:51:22.476058 8173 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40365 } }
I20260810 06:51:22.476480 8173 leader_election.cc:302] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aef36fe61d34b74acbac958b9bf8a1b; no voters:
I20260810 06:51:22.477001 8173 leader_election.cc:288] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:51:22.477119 8183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:51:22.477721 8183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [term 1 LEADER]: Becoming Leader. State: Replica: 1aef36fe61d34b74acbac958b9bf8a1b, State: Running, Role: LEADER
I20260810 06:51:22.478235 8183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40365 } }
I20260810 06:51:22.478777 8173 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:51:22.481150 8184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40365 } } }
I20260810 06:51:22.481611 8184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [sys.catalog]: This master's current role is: LEADER
I20260810 06:51:22.483093 8185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aef36fe61d34b74acbac958b9bf8a1b. Latest consensus state: current_term: 1 leader_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aef36fe61d34b74acbac958b9bf8a1b" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40365 } } }
I20260810 06:51:22.483484 8185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b [sys.catalog]: This master's current role is: LEADER
I20260810 06:51:22.487818 8195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:51:22.492882 8195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:51:22.500375 8195 catalog_manager.cc:1347] Generated new cluster ID: eae7917e838c42519bf87ea549457ad2
I20260810 06:51:22.500473 8195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:51:22.511123 8195 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:51:22.512888 8195 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:51:22.526284 8195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b: Generated new TSK 0
I20260810 06:51:22.527031 8195 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:51:22.594431 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40365
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:51:22.919095 8218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:22.919373 8218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:22.919509 8218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:22.926519 8218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:22.926636 8218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:22.926790 8218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:51:22.936100 8218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:22.937345 8218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:22.938568 8218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:22.951326 8230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:22.951290 8231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:22.951929 8238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:22.952378 8218 server_base.cc:1034] running on GCE node
I20260810 06:51:22.953156 8218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:22.954312 8218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:22.955554 8218 hybrid_clock.cc:648] HybridClock initialized: now 1786344682955478 us; error 44 us; skew 500 ppm
I20260810 06:51:22.956001 8218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:51:22 dist-test-slave-f7th krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344682, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:51:22.960796 8218 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:51:22.962322 8218 webserver.cc:460] Webserver started at http://127.22.11.129:35621/ using document root <none> and password file <none>
I20260810 06:51:22.963124 8218 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:22.963266 8218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:22.963604 8218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:22.966559 8218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "07fe12526e114d59b7343e5e83ef0497"
format_stamp: "Formatted at 2026-08-10 06:51:22 on dist-test-slave-f7th"
server_key: "e4b6aa020fbdfa561a9507ffceb8a427"
server_key_iv: "d119b10fde1df7c56bab7cffd515f9b7"
server_key_version: "encryptionkey@0"
I20260810 06:51:22.967276 8218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "07fe12526e114d59b7343e5e83ef0497"
format_stamp: "Formatted at 2026-08-10 06:51:22 on dist-test-slave-f7th"
server_key: "e4b6aa020fbdfa561a9507ffceb8a427"
server_key_iv: "d119b10fde1df7c56bab7cffd515f9b7"
server_key_version: "encryptionkey@0"
I20260810 06:51:22.972052 8218 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:51:22.975167 8250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:22.976368 8218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:51:22.976516 8218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "07fe12526e114d59b7343e5e83ef0497"
format_stamp: "Formatted at 2026-08-10 06:51:22 on dist-test-slave-f7th"
server_key: "e4b6aa020fbdfa561a9507ffceb8a427"
server_key_iv: "d119b10fde1df7c56bab7cffd515f9b7"
server_key_version: "encryptionkey@0"
I20260810 06:51:22.976725 8218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:22.988925 8218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:22.992331 8218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:22.992578 8218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:22.993486 8218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:22.994683 8218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:22.994753 8218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:22.994866 8218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:22.994923 8218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:23.023533 8218 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:42415
I20260810 06:51:23.023578 8467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:42415 every 8 connection(s)
I20260810 06:51:23.024718 8218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:51:23.029320 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 8218
I20260810 06:51:23.029521 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:51:23.030133 22574 external_mini_cluster.cc:1428] Setting key ce9c80282597d07c30bf2dd5e4928e0d
Aug 10 06:51:23 dist-test-slave-f7th krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344682, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:51:23.044587 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:23.027329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:43119 (local address 127.22.11.190:40365)
0810 06:51:23.027729 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:23.027734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:23.028660 (+ 926us) server_negotiation.cc:407] Connection header received
0810 06:51:23.030467 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:23.030473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:23.030577 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:23.030679 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:23.032417 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.033141 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.035226 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.035603 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.035690 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:23.039268 (+ 3578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:23.039272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:23.039276 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:23.039307 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:23.041764 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:23.042428 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:23.042432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:23.042434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:23.042560 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:23.042959 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:23.042965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:23.042967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:23.043222 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:23.043351 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:23.044000 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 06:51:23.044222 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":214,"threads_started":1}
I20260810 06:51:23.045459 8473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:23.027764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40365 (local address 127.22.11.129:43119)
0810 06:51:23.028493 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:23.028529 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:23.030211 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:23.030895 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:23.030903 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:23.031559 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:23.032243 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:23.032271 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.033311 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.033316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:23.035095 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:23.035111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.035806 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.035813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:23.036075 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:23.036736 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:23.036762 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:23.039070 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:23.041901 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:51:23.041910 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:51:23.041925 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:51:23.042317 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:51:23.042662 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:51:23.042667 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:51:23.042671 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:51:23.042855 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:51:23.043344 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:23.043351 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:23.043650 (+ 299us) client_negotiation.cc:769] Sending connection context
0810 06:51:23.043876 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 06:51:23.044360 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":293,"threads_started":1}
I20260810 06:51:23.046895 8469 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40365
I20260810 06:51:23.047159 8469 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:23.047834 8469 heartbeater.cc:511] Master 127.22.11.190:40365 requested a full tablet report, sending...
I20260810 06:51:23.050014 8108 ts_manager.cc:194] Registered new tserver with Master: 07fe12526e114d59b7343e5e83ef0497 (127.22.11.129:42415)
I20260810 06:51:23.051903 8108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:43119
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:51:23.088433 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40365
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:51:23.408083 8479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:23.408334 8479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:23.408432 8479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:23.415246 8479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:23.415336 8479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:23.415448 8479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:51:23.424423 8479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:23.425647 8479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:23.426811 8479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:23.438795 8485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:23.438969 8489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:23.439364 8492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:23.440310 8479 server_base.cc:1034] running on GCE node
I20260810 06:51:23.440986 8479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:23.442020 8479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:23.443245 8479 hybrid_clock.cc:648] HybridClock initialized: now 1786344683443193 us; error 38 us; skew 500 ppm
I20260810 06:51:23.443603 8479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:51:23 dist-test-slave-f7th krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344683, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:51:23.447219 8479 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:51:23.448735 8479 webserver.cc:460] Webserver started at http://127.22.11.130:33337/ using document root <none> and password file <none>
I20260810 06:51:23.449505 8479 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:23.449637 8479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:23.449954 8479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:23.453655 8479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "3ab6d25047d6466fbb28ca3e05e61cd3"
format_stamp: "Formatted at 2026-08-10 06:51:23 on dist-test-slave-f7th"
server_key: "c8c68adad74a2c23e579172aad3bb937"
server_key_iv: "262b24c52eb955da6c6d23e56d190a06"
server_key_version: "encryptionkey@0"
I20260810 06:51:23.454583 8479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "3ab6d25047d6466fbb28ca3e05e61cd3"
format_stamp: "Formatted at 2026-08-10 06:51:23 on dist-test-slave-f7th"
server_key: "c8c68adad74a2c23e579172aad3bb937"
server_key_iv: "262b24c52eb955da6c6d23e56d190a06"
server_key_version: "encryptionkey@0"
I20260810 06:51:23.460297 8479 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 06:51:23.463507 8514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:23.464764 8479 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:51:23.464892 8479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "3ab6d25047d6466fbb28ca3e05e61cd3"
format_stamp: "Formatted at 2026-08-10 06:51:23 on dist-test-slave-f7th"
server_key: "c8c68adad74a2c23e579172aad3bb937"
server_key_iv: "262b24c52eb955da6c6d23e56d190a06"
server_key_version: "encryptionkey@0"
I20260810 06:51:23.465039 8479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:23.476269 8479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:23.479803 8479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:23.480109 8479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:23.481014 8479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:23.482293 8479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:23.482363 8479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:23.482468 8479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:23.482523 8479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:23.512216 8479 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35315
I20260810 06:51:23.512259 8718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35315 every 8 connection(s)
I20260810 06:51:23.513404 8479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:51:23.520779 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 8479
I20260810 06:51:23.520954 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:51:23.521370 22574 external_mini_cluster.cc:1428] Setting key e2eca0f0fd600609cf533d008711931d
Aug 10 06:51:23 dist-test-slave-f7th krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344683, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:51:23.532301 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:23.515744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:32867 (local address 127.22.11.190:40365)
0810 06:51:23.515939 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:23.515944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:23.516986 (+ 1042us) server_negotiation.cc:407] Connection header received
0810 06:51:23.518754 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:23.518760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:23.518839 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:23.518926 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:23.520351 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.521189 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.522824 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.523356 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.523467 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:23.526734 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:23.526740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:23.526745 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:23.526781 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:23.529571 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:23.530205 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:23.530210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:23.530213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:23.530359 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:23.530731 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:23.530735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:23.530738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:23.531059 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:23.531194 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:23.531765 (+ 571us) server_negotiation.cc:299] Negotiation successful
0810 06:51:23.531986 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:51:23.532994 8722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:23.516116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40365 (local address 127.22.11.130:32867)
0810 06:51:23.516837 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:23.516873 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:23.518485 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:23.519162 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:23.519170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:23.519645 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:23.520180 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:23.520198 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.521332 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.521336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:23.522678 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:23.522695 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:23.523576 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:23.523584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:23.523786 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:23.524361 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:23.524383 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:23.526473 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:23.529692 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:51:23.529697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:51:23.529707 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:51:23.530100 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:51:23.530450 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:51:23.530454 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:51:23.530456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:51:23.530637 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:51:23.531154 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:23.531159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:23.531431 (+ 272us) client_negotiation.cc:769] Sending connection context
0810 06:51:23.531660 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 06:51:23.531993 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":267,"threads_started":1}
I20260810 06:51:23.534188 8720 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40365
I20260810 06:51:23.534461 8720 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:23.535116 8720 heartbeater.cc:511] Master 127.22.11.190:40365 requested a full tablet report, sending...
I20260810 06:51:23.536481 8108 ts_manager.cc:194] Registered new tserver with Master: 3ab6d25047d6466fbb28ca3e05e61cd3 (127.22.11.130:35315)
I20260810 06:51:23.537532 8108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:32867
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:51:23.579550 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40365
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:51:23.905345 8727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:51:23.905617 8727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:51:23.905750 8727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:51:23.912817 8727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:51:23.912918 8727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:51:23.913033 8727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:51:23.922199 8727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:51:23.923326 8727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:51:23.924542 8727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:51:23.937145 8742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:23.937182 8736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:51:23.937227 8735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:51:23.938033 8727 server_base.cc:1034] running on GCE node
I20260810 06:51:23.938588 8727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:51:23.939451 8727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:51:23.940671 8727 hybrid_clock.cc:648] HybridClock initialized: now 1786344683940625 us; error 45 us; skew 500 ppm
I20260810 06:51:23.940986 8727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:51:23 dist-test-slave-f7th krb5kdc[8023](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344683, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:51:23.943940 8727 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:51:23.945402 8727 webserver.cc:460] Webserver started at http://127.22.11.131:40755/ using document root <none> and password file <none>
I20260810 06:51:23.946044 8727 fs_manager.cc:359] Metadata directory not provided
I20260810 06:51:23.946161 8727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:51:23.946440 8727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:51:23.949239 8727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "ba86982184be4e77bd1468b59e4f325f"
format_stamp: "Formatted at 2026-08-10 06:51:23 on dist-test-slave-f7th"
server_key: "6dee8c6ba35e47249a6c5e6f9e869ab5"
server_key_iv: "32594d3190b2a37eaf1949712a22afdc"
server_key_version: "encryptionkey@0"
I20260810 06:51:23.949994 8727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "ba86982184be4e77bd1468b59e4f325f"
format_stamp: "Formatted at 2026-08-10 06:51:23 on dist-test-slave-f7th"
server_key: "6dee8c6ba35e47249a6c5e6f9e869ab5"
server_key_iv: "32594d3190b2a37eaf1949712a22afdc"
server_key_version: "encryptionkey@0"
I20260810 06:51:23.954805 8727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:51:23.958066 8751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:23.959309 8727 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:51:23.959458 8727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "ba86982184be4e77bd1468b59e4f325f"
format_stamp: "Formatted at 2026-08-10 06:51:23 on dist-test-slave-f7th"
server_key: "6dee8c6ba35e47249a6c5e6f9e869ab5"
server_key_iv: "32594d3190b2a37eaf1949712a22afdc"
server_key_version: "encryptionkey@0"
I20260810 06:51:23.959646 8727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:51:23.985368 8727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:51:23.989001 8727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:51:23.989257 8727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:51:23.990139 8727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:51:23.991385 8727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:51:23.991464 8727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:23.991581 8727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:51:23.991634 8727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:51:24.021903 8727 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:38833
I20260810 06:51:24.021941 8972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:38833 every 8 connection(s)
I20260810 06:51:24.023226 8727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:51:24.031533 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 8727
I20260810 06:51:24.031726 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:51:24.032166 22574 external_mini_cluster.cc:1428] Setting key 47c4a64189746d0eb0467445b4acb09f
Aug 10 06:51:24 dist-test-slave-f7th krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344683, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:51:24.042583 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.025793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:44319 (local address 127.22.11.190:40365)
0810 06:51:24.025954 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.025958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.027306 (+ 1348us) server_negotiation.cc:407] Connection header received
0810 06:51:24.029118 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.029124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.029204 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.029295 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:24.030721 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.031684 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.033504 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.034071 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.034195 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.037517 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:24.037523 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:24.037528 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:24.037569 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:24.039783 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.040446 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.040452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.040456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.040597 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.041020 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.041025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.041027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.041273 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:24.041402 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.042030 (+ 628us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.042243 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:51:24.043416 8981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.026396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40365 (local address 127.22.11.131:44319)
0810 06:51:24.027142 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:24.027180 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:24.028840 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:24.029518 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:24.029526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:24.030005 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:51:24.030553 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.030570 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.031856 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.031862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.033353 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.033377 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.034308 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.034315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.034539 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.035174 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:51:24.035197 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:51:24.037326 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:51:24.039907 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:51:24.039916 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:51:24.039930 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:51:24.040328 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:51:24.040718 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:51:24.040723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:51:24.040727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:51:24.040905 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:51:24.041394 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:51:24.041402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:51:24.041693 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 06:51:24.041916 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 06:51:24.042389 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":321,"threads_started":1}
I20260810 06:51:24.044590 8975 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40365
I20260810 06:51:24.044895 8975 heartbeater.cc:464] Registering TS with master...
I20260810 06:51:24.045635 8975 heartbeater.cc:511] Master 127.22.11.190:40365 requested a full tablet report, sending...
I20260810 06:51:24.047187 8105 ts_manager.cc:194] Registered new tserver with Master: ba86982184be4e77bd1468b59e4f325f (127.22.11.131:38833)
I20260810 06:51:24.048260 8105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:44319
I20260810 06:51:24.054629 8469 heartbeater.cc:503] Master 127.22.11.190:40365 was elected leader, sending a full tablet report...
I20260810 06:51:24.058955 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:51:24.074506 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.063376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47290 (local address 127.22.11.190:40365)
0810 06:51:24.063570 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.063577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.063972 (+ 395us) server_negotiation.cc:407] Connection header received
0810 06:51:24.064121 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.064125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.064180 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.064254 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:24.065684 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.066382 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.067498 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.067905 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.067988 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.070141 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:24.070146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:24.070149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:24.070177 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:24.072382 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.072991 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.072994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.072997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.073113 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.073390 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.073393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.073395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.073680 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:24.073849 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.074076 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.074224 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:51:24.078099 8105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:51:24.085122 8105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:51:24.108507 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.096786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42415 (local address 127.0.0.1:53260)
0810 06:51:24.097979 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:24.098010 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:24.098205 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:24.099265 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:24.099274 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:24.099308 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:51:24.099735 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.099747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.102392 (+ 2645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.102398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.104927 (+ 2529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.104946 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.107480 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.107487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.107714 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.107766 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:51:24.107920 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:51:24.108118 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"thread_start_us":876,"threads_started":1}
I20260810 06:51:24.109632 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.097005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.22.11.129:42415)
0810 06:51:24.097609 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.097620 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.098135 (+ 515us) server_negotiation.cc:407] Connection header received
0810 06:51:24.098478 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.098486 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.098757 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.098973 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:51:24.099963 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.102213 (+ 2250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.105140 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.106925 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.107065 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.107326 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.109092 (+ 1766us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.109282 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":318,"threads_started":1}
I20260810 06:51:24.114048 8351 tablet_service.cc:1467] Processing CreateTablet for tablet 0d872e4c27f0487683208efff06922b9 (DEFAULT_TABLE table=test-table [id=65af2af95f524d28a70776f822173a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:51:24.115645 8351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d872e4c27f0487683208efff06922b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:51:24.118724 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.096896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35315 (local address 127.0.0.1:48062)
0810 06:51:24.108786 (+ 11890us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:24.108805 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:24.108921 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:24.112056 (+ 3135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:24.112061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:24.112097 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:51:24.112286 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.112297 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.114330 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.114336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.116161 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.116178 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.117955 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.117961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.118181 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.118215 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:51:24.118296 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:51:24.118455 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11789,"mutex_wait_us":3,"threads_started":1}
I20260810 06:51:24.119671 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.097891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.22.11.130:35315)
0810 06:51:24.111453 (+ 13562us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.111461 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.111491 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:51:24.111587 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.111595 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.111823 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.112467 (+ 644us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:51:24.112723 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.114161 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.116385 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.117767 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.118958 (+ 1191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.119146 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.119212 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.119372 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13366,"thread_start_us":13229,"threads_started":1}
I20260810 06:51:24.121714 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.097435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:38833 (local address 127.0.0.1:56968)
0810 06:51:24.112276 (+ 14841us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:24.112296 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:24.112422 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:24.113404 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:24.113409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:24.113434 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:51:24.113850 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.113863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.115946 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.115952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.118506 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.118521 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.120932 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.120939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.121103 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.121133 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:51:24.121213 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:51:24.121334 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14693,"mutex_wait_us":295,"thread_start_us":14564,"threads_started":1}
I20260810 06:51:24.121731 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.098533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56968 (local address 127.22.11.131:38833)
0810 06:51:24.111818 (+ 13285us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.111827 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.112549 (+ 722us) server_negotiation.cc:407] Connection header received
0810 06:51:24.112682 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.112690 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.112925 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.113119 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:51:24.114072 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.115761 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.118690 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.120746 (+ 2056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.120869 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.121078 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.121277 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.121425 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13102,"thread_start_us":20951,"threads_started":1}
I20260810 06:51:24.123175 8605 tablet_service.cc:1467] Processing CreateTablet for tablet 0d872e4c27f0487683208efff06922b9 (DEFAULT_TABLE table=test-table [id=65af2af95f524d28a70776f822173a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:51:24.124541 8605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d872e4c27f0487683208efff06922b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:51:24.124928 8850 tablet_service.cc:1467] Processing CreateTablet for tablet 0d872e4c27f0487683208efff06922b9 (DEFAULT_TABLE table=test-table [id=65af2af95f524d28a70776f822173a19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:51:24.126011 8850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d872e4c27f0487683208efff06922b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:51:24.130046 9016 tablet_bootstrap.cc:492] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Bootstrap starting.
I20260810 06:51:24.133944 9016 tablet_bootstrap.cc:654] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Neither blocks nor log segments found. Creating new log.
I20260810 06:51:24.135169 9016 log.cc:824] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Log is configured to *not* fsync() on all Append() calls
I20260810 06:51:24.137221 9018 tablet_bootstrap.cc:492] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Bootstrap starting.
I20260810 06:51:24.138335 9016 tablet_bootstrap.cc:492] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: No bootstrap required, opened a new log
I20260810 06:51:24.138568 9019 tablet_bootstrap.cc:492] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Bootstrap starting.
I20260810 06:51:24.138722 9016 ts_tablet_manager.cc:1397] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260810 06:51:24.141165 9018 tablet_bootstrap.cc:654] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Neither blocks nor log segments found. Creating new log.
I20260810 06:51:24.141937 9019 tablet_bootstrap.cc:654] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Neither blocks nor log segments found. Creating new log.
I20260810 06:51:24.142457 9018 log.cc:824] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Log is configured to *not* fsync() on all Append() calls
I20260810 06:51:24.143124 9019 log.cc:824] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:51:24.145505 9018 tablet_bootstrap.cc:492] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: No bootstrap required, opened a new log
I20260810 06:51:24.145787 9018 ts_tablet_manager.cc:1397] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260810 06:51:24.146672 9016 raft_consensus.cc:348] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.147014 9016 raft_consensus.cc:374] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:51:24.147130 9016 raft_consensus.cc:729] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fe12526e114d59b7343e5e83ef0497, State: Initialized, Role: FOLLOWER
I20260810 06:51:24.147753 9016 consensus_queue.cc:260] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.147966 9019 tablet_bootstrap.cc:492] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: No bootstrap required, opened a new log
I20260810 06:51:24.148247 9019 ts_tablet_manager.cc:1397] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260810 06:51:24.150241 9016 ts_tablet_manager.cc:1428] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Time spent starting tablet: real 0.011s user 0.007s sys 0.002s
I20260810 06:51:24.155119 9018 raft_consensus.cc:348] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.155457 9018 raft_consensus.cc:374] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:51:24.155543 9018 raft_consensus.cc:729] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab6d25047d6466fbb28ca3e05e61cd3, State: Initialized, Role: FOLLOWER
I20260810 06:51:24.156033 9018 consensus_queue.cc:260] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.157136 8720 heartbeater.cc:503] Master 127.22.11.190:40365 was elected leader, sending a full tablet report...
I20260810 06:51:24.157035 9019 raft_consensus.cc:348] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.157330 9019 raft_consensus.cc:374] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:51:24.157428 9019 raft_consensus.cc:729] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba86982184be4e77bd1468b59e4f325f, State: Initialized, Role: FOLLOWER
I20260810 06:51:24.157639 9018 ts_tablet_manager.cc:1428] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260810 06:51:24.157948 9019 consensus_queue.cc:260] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.158937 8975 heartbeater.cc:503] Master 127.22.11.190:40365 was elected leader, sending a full tablet report...
I20260810 06:51:24.159343 9019 ts_tablet_manager.cc:1428] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
W20260810 06:51:24.265223 8721 tablet.cc:2369] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:51:24.274818 8978 tablet.cc:2369] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:51:24.276837 8470 tablet.cc:2369] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:51:24.352376 9033 raft_consensus.cc:482] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:51:24.352705 9033 raft_consensus.cc:504] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.354446 9033 leader_election.cc:288] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3ab6d25047d6466fbb28ca3e05e61cd3 (127.22.11.130:35315), ba86982184be4e77bd1468b59e4f325f (127.22.11.131:38833)
I20260810 06:51:24.362587 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.354911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:42257 (local address 127.22.11.130:35315)
0810 06:51:24.355133 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.355139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.355736 (+ 597us) server_negotiation.cc:407] Connection header received
0810 06:51:24.355939 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.355945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.356006 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.356096 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:51:24.356731 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.357767 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.360350 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.361584 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.361717 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.361790 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.362213 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.362337 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:51:24.362603 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.354940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:58465 (local address 127.22.11.131:38833)
0810 06:51:24.355118 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.355123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.356102 (+ 979us) server_negotiation.cc:407] Connection header received
0810 06:51:24.356252 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.356256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.356313 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.356377 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:51:24.356972 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.357997 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.360347 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.361566 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.361728 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.361790 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.362209 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.362337 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:51:24.362790 9040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.354911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35315 (local address 127.22.11.129:42257)
0810 06:51:24.355628 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:24.355645 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:24.355800 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:24.356132 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:24.356137 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:24.356159 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:51:24.356568 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.356584 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.357891 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.357894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.360206 (+ 2312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.360221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.361842 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.361848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.362025 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.362081 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 06:51:24.362170 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 06:51:24.362356 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":96,"thread_start_us":388,"threads_started":1}
I20260810 06:51:24.362869 9041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.354925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:38833 (local address 127.22.11.129:58465)
0810 06:51:24.355989 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0810 06:51:24.356004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:51:24.356127 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:51:24.356435 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:51:24.356439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:51:24.356468 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:51:24.356819 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.356828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.358111 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.358116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.360206 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:51:24.360221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.361789 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.361796 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:51:24.361980 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.362079 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 06:51:24.362173 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 06:51:24.362356 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":972,"mutex_wait_us":356,"thread_start_us":489,"threads_started":1}
I20260810 06:51:24.363965 8892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d872e4c27f0487683208efff06922b9" candidate_uuid: "07fe12526e114d59b7343e5e83ef0497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba86982184be4e77bd1468b59e4f325f" is_pre_election: true
I20260810 06:51:24.363965 8642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d872e4c27f0487683208efff06922b9" candidate_uuid: "07fe12526e114d59b7343e5e83ef0497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" is_pre_election: true
I20260810 06:51:24.364566 8892 raft_consensus.cc:2454] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07fe12526e114d59b7343e5e83ef0497 in term 0.
I20260810 06:51:24.364555 8642 raft_consensus.cc:2454] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07fe12526e114d59b7343e5e83ef0497 in term 0.
I20260810 06:51:24.365267 8256 leader_election.cc:302] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fe12526e114d59b7343e5e83ef0497, 3ab6d25047d6466fbb28ca3e05e61cd3; no voters:
I20260810 06:51:24.365684 9033 raft_consensus.cc:2781] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:51:24.365831 9033 raft_consensus.cc:482] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:51:24.365926 9033 raft_consensus.cc:3037] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:51:24.368089 9033 raft_consensus.cc:504] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.368733 9033 leader_election.cc:288] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [CANDIDATE]: Term 1 election: Requested vote from peers 3ab6d25047d6466fbb28ca3e05e61cd3 (127.22.11.130:35315), ba86982184be4e77bd1468b59e4f325f (127.22.11.131:38833)
I20260810 06:51:24.369119 8642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d872e4c27f0487683208efff06922b9" candidate_uuid: "07fe12526e114d59b7343e5e83ef0497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3"
I20260810 06:51:24.369120 8892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d872e4c27f0487683208efff06922b9" candidate_uuid: "07fe12526e114d59b7343e5e83ef0497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba86982184be4e77bd1468b59e4f325f"
I20260810 06:51:24.369283 8892 raft_consensus.cc:3037] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:51:24.369279 8642 raft_consensus.cc:3037] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:51:24.371366 8892 raft_consensus.cc:2454] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fe12526e114d59b7343e5e83ef0497 in term 1.
I20260810 06:51:24.371403 8642 raft_consensus.cc:2454] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07fe12526e114d59b7343e5e83ef0497 in term 1.
I20260810 06:51:24.371711 8255 leader_election.cc:302] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07fe12526e114d59b7343e5e83ef0497, ba86982184be4e77bd1468b59e4f325f; no voters:
I20260810 06:51:24.371966 9033 raft_consensus.cc:2781] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:51:24.372354 9033 raft_consensus.cc:686] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 1 LEADER]: Becoming Leader. State: Replica: 07fe12526e114d59b7343e5e83ef0497, State: Running, Role: LEADER
I20260810 06:51:24.372810 9033 consensus_queue.cc:237] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } }
I20260810 06:51:24.377035 8108 catalog_manager.cc:5495] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07fe12526e114d59b7343e5e83ef0497 (127.22.11.129). New cstate: current_term: 1 leader_uuid: "07fe12526e114d59b7343e5e83ef0497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07fe12526e114d59b7343e5e83ef0497" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 } health_report { overall_health: UNKNOWN } } }
I20260810 06:51:24.402592 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.397906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.22.11.129:42415)
0810 06:51:24.398065 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.398070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.398189 (+ 119us) server_negotiation.cc:407] Connection header received
0810 06:51:24.398354 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.398358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.398430 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.398558 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:51:24.398827 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.399529 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.400850 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.401259 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.401347 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.401712 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:51:24.402190 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:51:24.402295 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.402332 (+ 37us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.402393 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:51:24.410144 8892 raft_consensus.cc:1261] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 1 FOLLOWER]: Refusing update from remote peer 07fe12526e114d59b7343e5e83ef0497: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:51:24.410147 8642 raft_consensus.cc:1261] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 1 FOLLOWER]: Refusing update from remote peer 07fe12526e114d59b7343e5e83ef0497: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:51:24.411051 9042 consensus_queue.cc:1035] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba86982184be4e77bd1468b59e4f325f" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 38833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:51:24.411396 9033 consensus_queue.cc:1035] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ab6d25047d6466fbb28ca3e05e61cd3" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:51:24.432480 8108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:51:24.432813 8108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:51:24.436281 8108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1aef36fe61d34b74acbac958b9bf8a1b: Sending DeleteTablet for 3 replicas of tablet 0d872e4c27f0487683208efff06922b9
I20260810 06:51:24.437340 8351 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d872e4c27f0487683208efff06922b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:51:24 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:53260
I20260810 06:51:24.437423 8605 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d872e4c27f0487683208efff06922b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:51:24 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:48062
I20260810 06:51:24.437515 8850 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d872e4c27f0487683208efff06922b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:51:24 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:56968
I20260810 06:51:24.441891 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 8218
I20260810 06:51:24.449026 9068 tablet_replica.cc:330] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: stopping tablet replica
I20260810 06:51:24.449440 9068 raft_consensus.cc:2229] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:51:24.449680 9066 tablet_replica.cc:330] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: stopping tablet replica
I20260810 06:51:24.450062 9068 raft_consensus.cc:2258] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:51:24.450548 9066 raft_consensus.cc:2229] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:51:24.450846 9064 tablet_replica.cc:330] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: stopping tablet replica
I20260810 06:51:24.451232 9066 raft_consensus.cc:2258] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:51:24.452927 9068 ts_tablet_manager.cc:1905] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:51:24.453372 9064 raft_consensus.cc:2229] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:51:24.453526 9066 ts_tablet_manager.cc:1905] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:51:24.454089 9064 raft_consensus.cc:2258] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:51:24.460296 9066 ts_tablet_manager.cc:1918] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:51:24.460522 9066 log.cc:1196] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/0d872e4c27f0487683208efff06922b9
I20260810 06:51:24.461081 9066 ts_tablet_manager.cc:1939] T 0d872e4c27f0487683208efff06922b9 P 3ab6d25047d6466fbb28ca3e05e61cd3: Deleting consensus metadata
I20260810 06:51:24.462496 9064 ts_tablet_manager.cc:1905] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:51:24.462874 8084 catalog_manager.cc:4841] TS 3ab6d25047d6466fbb28ca3e05e61cd3 (127.22.11.130:35315): tablet 0d872e4c27f0487683208efff06922b9 (table test-table [id=65af2af95f524d28a70776f822173a19]) successfully deleted
Aug 10 06:51:24 dist-test-slave-f7th krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:51:24.469257 9068 ts_tablet_manager.cc:1918] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:51:24.469755 9068 log.cc:1196] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/0d872e4c27f0487683208efff06922b9
I20260810 06:51:24.470424 9068 ts_tablet_manager.cc:1939] T 0d872e4c27f0487683208efff06922b9 P ba86982184be4e77bd1468b59e4f325f: Deleting consensus metadata
I20260810 06:51:24.472265 8083 catalog_manager.cc:4841] TS ba86982184be4e77bd1468b59e4f325f (127.22.11.131:38833): tablet 0d872e4c27f0487683208efff06922b9 (table test-table [id=65af2af95f524d28a70776f822173a19]) successfully deleted
I20260810 06:51:24.474687 9064 ts_tablet_manager.cc:1918] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:51:24.474895 9064 log.cc:1196] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/0d872e4c27f0487683208efff06922b9
I20260810 06:51:24.475409 9064 ts_tablet_manager.cc:1939] T 0d872e4c27f0487683208efff06922b9 P 07fe12526e114d59b7343e5e83ef0497: Deleting consensus metadata
I20260810 06:51:24.475878 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.443375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.22.11.129:42415)
0810 06:51:24.443555 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.443562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.450285 (+ 6723us) server_negotiation.cc:407] Connection header received
0810 06:51:24.450493 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.450498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.450610 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.450743 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:24.452257 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.453477 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.458613 (+ 5136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.459321 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.459420 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.467420 (+ 8000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:24.467435 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:24.467453 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:24.467504 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:24.471507 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.473076 (+ 1569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.473085 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.473088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.473278 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.473778 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.473783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.473787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.474252 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:24.474543 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.475225 (+ 682us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.475510 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:51:24.476995 8078 catalog_manager.cc:4841] TS 07fe12526e114d59b7343e5e83ef0497 (127.22.11.129:42415): tablet 0d872e4c27f0487683208efff06922b9 (table test-table [id=65af2af95f524d28a70776f822173a19]) successfully deleted
I20260810 06:51:24.478222 8462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:24.576112 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 8218
I20260810 06:51:24.600337 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 8479
Aug 10 06:51:24 dist-test-slave-f7th krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:51:24.614244 9011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.601162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.22.11.130:35315)
0810 06:51:24.601346 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.601352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.601372 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:51:24.601484 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.601489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.601565 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.601651 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:24.602880 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.603636 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.605090 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.605503 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.605617 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.609452 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:24.609465 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:24.609481 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:24.609518 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:24.612008 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.612566 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.612572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.612574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.612722 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.613005 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.613008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.613010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.613402 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:24.613623 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.613827 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.613984 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:51:24.614897 8712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:24.702437 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 8479
I20260810 06:51:24.728574 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 8727
Aug 10 06:51:24 dist-test-slave-f7th krb5kdc[8023](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:51:24.742643 9013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:51:24.729382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56976 (local address 127.22.11.131:38833)
0810 06:51:24.729567 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:51:24.729573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:51:24.729593 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:51:24.729769 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:51:24.729774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:51:24.729851 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:51:24.729939 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:51:24.731221 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.731913 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.733236 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:51:24.733629 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:51:24.733741 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:51:24.737762 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:51:24.737777 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:51:24.737795 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:51:24.737833 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:51:24.740439 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.741069 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.741077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.741079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.741222 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:51:24.741496 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:51:24.741499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:51:24.741501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:51:24.741812 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:51:24.742040 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:51:24.742226 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:51:24.742385 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:51:24.743218 8963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:24.830480 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 8727
I20260810 06:51:24.850242 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 8047
I20260810 06:51:24.851332 8159 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:51:24.958045 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 8047
2026-08-10T06:51:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4438 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:51:25 dist-test-slave-f7th krb5kdc[9081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:51:25 dist-test-slave-f7th krb5kdc[9081](info): set up 2 sockets
Aug 10 06:51:25 dist-test-slave-f7th krb5kdc[9081](info): commencing operation
krb5kdc: starting...
W20260810 06:51:26.207490 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:51:26 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:51:26Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/rangeradmin_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/rangeradmin_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/rangerlookup_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/rangerlookup_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
I20260810 06:51:26.595407 22574 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJIbnRf/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/postgres -l logfile start
2026-08-10 06:51:28.917 UTC [9141] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:51:28.917 UTC [9141] LOG: listening on IPv4 address "127.22.11.148", port 58907
2026-08-10 06:51:28.920 UTC [9141] LOG: listening on Unix socket "/tmp/.s.PGSQL.58907"
2026-08-10 06:51:28.941 UTC [9146] LOG: database system was shut down at 2026-08-10 06:51:27 UTC
2026-08-10 06:51:28.944 UTC [9141] LOG: database system is ready to accept connections
I20260810 06:51:30.065168 22574 mini_postgres.cc:96] Postgres bound to 58907
2026-08-10 06:51:30.089 UTC [9154] FATAL: database "slave" does not exist
127.22.11.148:58907 - accepting connections
I20260810 06:51:30.090562 22574 mini_ranger.cc:161] Starting Ranger...
I20260810 06:51:30.119911 22574 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:51:30.516346 22574 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:51:30.516485 22574 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-admin
2026-08-10 06:51:30,955 [I] DB FLAVOR :POSTGRES
2026-08-10 06:51:30,955 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:51:30,955 [I] Checking connection..
2026-08-10 06:51:30,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:51:31,220 [I] Checking connection passed.
2026-08-10 06:51:31,220 [I] --------- Verifying version history table ---------
2026-08-10 06:51:31,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:51:31,478 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:51:31,478 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:51:31,721 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:51:31,721 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:51:31,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:51:31.968 UTC [9268] WARNING: there is no transaction in progress
2026-08-10 06:51:31,974 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:51:31,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:51:32,228 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:51:32,228 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:51:32,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:51:32,482 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:51:32,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-10 06:51:32,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:51:33,231 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:51:33,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:51:33,489 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:51:33,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:51:33,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:51:33,952 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:51:33,952 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:51:34.843 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.858 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.874 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.887 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.900 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.916 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.965 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.971 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.983 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:34.992 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:35.002 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:35.014 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:35.023 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:35.028 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:35.034 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:35.039 UTC [9443] WARNING: there is no transaction in progress
2026-08-10 06:51:35,568 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:51:35,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:51:35,815 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:51:35,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:51:36,071 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:51:36,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:51:36,300 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:51:36,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:51:36,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:51:36,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-10 06:51:37,024 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:51:37,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:51:37,280 [I] DB_PATCHES have already been applied
I20260810 06:51:37.287441 22574 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJIbnRf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:51:37.287557 22574 mini_ranger.cc:193] Using host: 127.22.11.148
I20260810 06:51:37.308694 22574 mini_ranger.cc:227] Ranger admin URL: http://127.22.11.148:59553
Aug 10, 2026 6:51:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:51:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:51:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:51:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:51:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:51:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:51:38 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:51:38.654822 22574 mini_ranger.cc:160] Time spent starting Ranger: real 8.564s user 0.000s sys 0.060s
Aug 10, 2026 6:51:57 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:51:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:51:58 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:51:58 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:51:58 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:51:59.474984 22574 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/rangerkms_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/rangerkms_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.11.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.11.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:51:59 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344719, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:51:59.671361 22574 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:51:59.694410 22574 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:52:00.352447 22574 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:52:00.352662 22574 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms
I20260810 06:52:00.352802 22574 mini_ranger_kms.cc:227] Using postgres at 127.22.11.148:58907
2026-08-10 06:52:00,833 [I] DB FLAVOR :POSTGRES
2026-08-10 06:52:00,834 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:52:00,834 [I] Checking connection
2026-08-10 06:52:00,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:52:01,062 [I] connection success
2026-08-10 06:52:01,062 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:52:01,063 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:52:01,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:52:01,309 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:52:01,309 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:52:01,309 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:52:01,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:58907/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:52:01,558 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:52:01.986632 22574 mini_ranger_kms.cc:310] Created kms service
I20260810 06:52:02.186017 22574 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:52:02.230101 22574 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:52:02 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344719, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:02.610862 22574 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:52:02.612152 22574 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:52:02.612229 22574 mini_ranger_kms.cc:240] Using host: 127.22.11.148
I20260810 06:52:02.638878 22574 mini_ranger_kms.cc:276] Ranger KMS PID: 9775
I20260810 06:52:02.639088 22574 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.11.148:59763
Aug 10, 2026 6:52:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:52:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:52:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:52:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:52:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:52:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:52:04 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344724, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.11.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:05.321591 22574 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.650s user 0.095s sys 0.429s
I20260810 06:52:05.321954 22574 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:52:05.322186 22574 mini_ranger_kms.cc:401] 127.22.11.148:59763/kms/v1/keys
Aug 10 06:52:05 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344724, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.11.148@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
Aug 10 06:52:05 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344724, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.11.148@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
Aug 10 06:52:08 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:52:08.262185 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:44431
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:44431
--ranger_config_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:59763/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:52:08.785990 9865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:08.786352 9865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:08.786537 9865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:08.797266 9865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:52:08.797415 9865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:08.797521 9865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:08.797657 9865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:52:08.797771 9865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:52:08.811447 9865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.11.148:59763/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:44431
--ranger_config_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:44431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:08.812990 9865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:08.814512 9865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:08.830437 9873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:08.830722 9872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:08.830564 9878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:08.832003 9865 server_base.cc:1034] running on GCE node
I20260810 06:52:08.833024 9865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:08.834481 9865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:08.835706 9865 hybrid_clock.cc:648] HybridClock initialized: now 1786344728835637 us; error 37 us; skew 500 ppm
I20260810 06:52:08.836020 9865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:08 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344728, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:08.844453 9865 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:52:08.846199 9865 webserver.cc:460] Webserver started at http://127.22.11.190:37373/ using document root <none> and password file <none>
I20260810 06:52:08.846903 9865 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:08.847051 9865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:08.847350 9865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:08 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344728, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:08.901917 9865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "455ac090eb7c4415ba2dbb105e55b1ee"
format_stamp: "Formatted at 2026-08-10 06:52:08 on dist-test-slave-f7th"
server_key: "9d371833fb74ccd05302b48f67938d71"
server_key_iv: "3d1d841feea27d23e608e7f9fcbe3625"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:08.903225 9865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "455ac090eb7c4415ba2dbb105e55b1ee"
format_stamp: "Formatted at 2026-08-10 06:52:08 on dist-test-slave-f7th"
server_key: "9d371833fb74ccd05302b48f67938d71"
server_key_iv: "3d1d841feea27d23e608e7f9fcbe3625"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:08.910369 9865 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260810 06:52:08.928979 9889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:08.930490 9865 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:52:08.930694 9865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "455ac090eb7c4415ba2dbb105e55b1ee"
format_stamp: "Formatted at 2026-08-10 06:52:08 on dist-test-slave-f7th"
server_key: "9d371833fb74ccd05302b48f67938d71"
server_key_iv: "3d1d841feea27d23e608e7f9fcbe3625"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:08.930964 9865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:52:08.946957 9865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:52:08.951840 9865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:52:08.952309 9865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:52:08.991684 9865 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:44431
I20260810 06:52:08.991848 9988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:44431 every 8 connection(s)
I20260810 06:52:08.993342 9865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:52:08.998474 9989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:52:09.003692 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 9865
I20260810 06:52:09.003875 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:52:09.004655 22574 external_mini_cluster.cc:1428] Setting key b71d3219d15ee6fa79289ea54db9a75b
I20260810 06:52:09.012356 9989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee: Bootstrap starting.
I20260810 06:52:09.017584 9989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee: Neither blocks nor log segments found. Creating new log.
I20260810 06:52:09.019622 9989 log.cc:824] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:52:09 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:52:09.024056 9989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee: No bootstrap required, opened a new log
I20260810 06:52:09.031072 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:09.006743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.22.11.190:44431)
0810 06:52:09.008189 (+ 1446us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:09.008203 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:09.008243 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:52:09.011210 (+ 2967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:09.011234 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:09.011654 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:09.011982 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:09.013669 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:09.015567 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:09.017291 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:09.018157 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:09.018283 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:09.022706 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:09.022717 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:09.022735 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:09.022779 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:09.026766 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:09.027375 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:09.027383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:09.027391 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:09.027584 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:09.027942 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:09.027947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:09.027950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:09.028727 (+ 777us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:09.029014 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:09.029365 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 06:52:09.029786 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1252,"thread_start_us":930,"threads_started":1}
I20260810 06:52:09.033849 9989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44431 } }
I20260810 06:52:09.034246 9989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:52:09.034341 9989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 455ac090eb7c4415ba2dbb105e55b1ee, State: Initialized, Role: FOLLOWER
I20260810 06:52:09.034842 9989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44431 } }
I20260810 06:52:09.035058 9989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:52:09.035164 9989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:52:09.035326 9989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:52:09.037716 9989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44431 } }
I20260810 06:52:09.038268 9989 leader_election.cc:302] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 455ac090eb7c4415ba2dbb105e55b1ee; no voters:
I20260810 06:52:09.039258 9989 leader_election.cc:288] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:52:09.039474 9997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:52:09.041023 9997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [term 1 LEADER]: Becoming Leader. State: Replica: 455ac090eb7c4415ba2dbb105e55b1ee, State: Running, Role: LEADER
I20260810 06:52:09.041754 9997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44431 } }
I20260810 06:52:09.042167 9989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:52:09.045588 10001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 455ac090eb7c4415ba2dbb105e55b1ee. Latest consensus state: current_term: 1 leader_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44431 } } }
I20260810 06:52:09.046736 10001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [sys.catalog]: This master's current role is: LEADER
I20260810 06:52:09.046389 10000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "455ac090eb7c4415ba2dbb105e55b1ee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44431 } } }
I20260810 06:52:09.047156 10000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee [sys.catalog]: This master's current role is: LEADER
I20260810 06:52:09.049286 10009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:52:09.055119 10009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:52:09.059240 9989 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:52:09.087086 10009 catalog_manager.cc:1347] Generated new cluster ID: 7edebc75c47f45adad5c2cb426f2f43e
I20260810 06:52:09.087297 10009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:52:09.124542 10009 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:52:09.127116 10009 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:52:09.147008 10009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee: Generated new TSK 0
I20260810 06:52:09.148306 10009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:52:09.956118 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:59763/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44431
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:52:10 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344730, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 06:52:10.498957 10055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:10.499416 10055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:10.499577 10055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:10.507402 10055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:10.507525 10055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:10.507685 10055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:52:10.521229 10055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.11.148:59763/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:10.522598 10055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:10.523952 10055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:10.550071 10069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:10.551540 10070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:10.561079 10084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:10.562433 10055 server_base.cc:1034] running on GCE node
I20260810 06:52:10.563201 10055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:10.564478 10055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:10.565718 10055 hybrid_clock.cc:648] HybridClock initialized: now 1786344730565606 us; error 102 us; skew 500 ppm
I20260810 06:52:10.566056 10055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:10 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344730, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:10.570686 10055 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:52:10.578855 10055 webserver.cc:460] Webserver started at http://127.22.11.129:37245/ using document root <none> and password file <none>
I20260810 06:52:10.579905 10055 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:10.580108 10055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:10.580600 10055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:10 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344730, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:10.607179 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "2cbeef5980dc4de18d0a3789e42ed307"
format_stamp: "Formatted at 2026-08-10 06:52:10 on dist-test-slave-f7th"
server_key: "c27ce6e184203ca68fd721a47c76008d"
server_key_iv: "0d9751a7c3906c42cc7e9464a476090f"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:10.609074 10055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "2cbeef5980dc4de18d0a3789e42ed307"
format_stamp: "Formatted at 2026-08-10 06:52:10 on dist-test-slave-f7th"
server_key: "c27ce6e184203ca68fd721a47c76008d"
server_key_iv: "0d9751a7c3906c42cc7e9464a476090f"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:10.618723 10055 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.005s
I20260810 06:52:10.646775 10099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:10.649121 10055 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.004s
I20260810 06:52:10.649398 10055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "2cbeef5980dc4de18d0a3789e42ed307"
format_stamp: "Formatted at 2026-08-10 06:52:10 on dist-test-slave-f7th"
server_key: "c27ce6e184203ca68fd721a47c76008d"
server_key_iv: "0d9751a7c3906c42cc7e9464a476090f"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:10.649807 10055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 06:52:10 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344730, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:10.721699 10055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:52:10.728840 10055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:52:10.729526 10055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:52:10.749167 10055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:52:10.769738 10055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:52:10.769981 10055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 06:52:10.770268 10055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:52:10.770478 10055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:10.863621 10055 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:36031
I20260810 06:52:10.863646 10306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:36031 every 8 connection(s)
I20260810 06:52:10.865417 10055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:52:10.867064 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 10055
I20260810 06:52:10.867277 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:52:10.868166 22574 external_mini_cluster.cc:1428] Setting key e856cccbae0a168ca5fd0b8e565c2aa7
Aug 10 06:52:10 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344730, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
I20260810 06:52:10.921563 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:10.873103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:55687 (local address 127.22.11.190:44431)
0810 06:52:10.893335 (+ 20232us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:10.893341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:10.893372 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:52:10.893436 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:10.893441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:10.893525 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:10.893653 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:10.896406 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:10.897567 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:10.900580 (+ 3013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:10.901452 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:10.901579 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:10.911188 (+ 9609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:10.911195 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:10.911201 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:10.911244 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:10.915759 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:10.916782 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:10.916788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:10.916791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:10.916984 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:10.917540 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:10.917545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:10.917549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:10.917922 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:10.918053 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:10.920930 (+ 2877us) server_negotiation.cc:299] Negotiation successful
0810 06:52:10.921172 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20100,"thread_start_us":19762,"threads_started":1}
I20260810 06:52:10.922154 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:10.871648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:44431 (local address 127.22.11.129:55687)
0810 06:52:10.883119 (+ 11471us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:10.883170 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:10.887245 (+ 4075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:10.893980 (+ 6735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:10.893993 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:10.894936 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:52:10.896179 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:10.896207 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:10.897765 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:10.897771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:10.900355 (+ 2584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:10.900382 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:10.902060 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:10.902069 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:10.902493 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:10.903502 (+ 1009us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:52:10.903534 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:52:10.907306 (+ 3772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:52:10.915945 (+ 8639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:10.915955 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:10.915972 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:10.916597 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:10.917129 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:10.917133 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:10.917136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:10.917402 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:10.918138 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:52:10.918146 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:52:10.918601 (+ 455us) client_negotiation.cc:769] Sending connection context
0810 06:52:10.918937 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 06:52:10.919485 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11147,"thread_start_us":13734,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
I20260810 06:52:10.924228 10310 heartbeater.cc:347] Connected to a master server at 127.22.11.190:44431
I20260810 06:52:10.924551 10310 heartbeater.cc:464] Registering TS with master...
I20260810 06:52:10.925455 10310 heartbeater.cc:511] Master 127.22.11.190:44431 requested a full tablet report, sending...
I20260810 06:52:10.928025 9923 ts_manager.cc:194] Registered new tserver with Master: 2cbeef5980dc4de18d0a3789e42ed307 (127.22.11.129:36031)
I20260810 06:52:10.930112 9923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:55687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:52:10.963543 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:59763/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44431
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:52:11 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344730, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
W20260810 06:52:11.510085 10318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:11.510462 10318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:11.510685 10318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:11.520082 10318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:11.520253 10318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:11.520471 10318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:52:11.533200 10318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.11.148:59763/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:11.534634 10318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:11.536124 10318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:11.553835 10337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:11.555150 10338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:11.555150 10341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:11.555769 10318 server_base.cc:1034] running on GCE node
I20260810 06:52:11.556496 10318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:11.557499 10318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:11.558699 10318 hybrid_clock.cc:648] HybridClock initialized: now 1786344731558653 us; error 33 us; skew 500 ppm
I20260810 06:52:11.558986 10318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:11 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344731, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:11.561986 10318 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:52:11.563419 10318 webserver.cc:460] Webserver started at http://127.22.11.130:35497/ using document root <none> and password file <none>
I20260810 06:52:11.564054 10318 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:11.564153 10318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:11.564400 10318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:11 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344731, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:11.580636 10318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "0e61fbb77c084f3682f552a4f5e1754f"
format_stamp: "Formatted at 2026-08-10 06:52:11 on dist-test-slave-f7th"
server_key: "4e7d72bdf17b1dacb8427195d3417f7c"
server_key_iv: "699f73a76d6d96c2adf7e996734b9ab0"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:11.581607 10318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "0e61fbb77c084f3682f552a4f5e1754f"
format_stamp: "Formatted at 2026-08-10 06:52:11 on dist-test-slave-f7th"
server_key: "4e7d72bdf17b1dacb8427195d3417f7c"
server_key_iv: "699f73a76d6d96c2adf7e996734b9ab0"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:11.587309 10318 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 06:52:11.603434 10361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:11.604938 10318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:52:11.605124 10318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "0e61fbb77c084f3682f552a4f5e1754f"
format_stamp: "Formatted at 2026-08-10 06:52:11 on dist-test-slave-f7th"
server_key: "4e7d72bdf17b1dacb8427195d3417f7c"
server_key_iv: "699f73a76d6d96c2adf7e996734b9ab0"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:11.605357 10318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:52:11.638021 10318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:52:11.642032 10318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:52:11.642356 10318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:52:11.643625 10318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:52:11.645460 10318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:52:11.645545 10318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:11.645757 10318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:52:11.645821 10318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:11.691131 10318 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35995
I20260810 06:52:11.691179 10549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35995 every 8 connection(s)
I20260810 06:52:11.692380 10318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:52:11.694682 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 10318
I20260810 06:52:11.694866 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:52:11.695441 22574 external_mini_cluster.cc:1428] Setting key 64575897db51378692685bbff96b5556
Aug 10 06:52:11 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344731, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:52:11.712175 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:11.694988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:35013 (local address 127.22.11.190:44431)
0810 06:52:11.695445 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:11.695449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:11.698697 (+ 3248us) server_negotiation.cc:407] Connection header received
0810 06:52:11.698749 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:11.698752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:11.698821 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:11.698910 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:11.700329 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:11.701123 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:11.702615 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:11.703009 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:11.703117 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:11.706378 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:11.706383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:11.706386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:11.706415 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:11.709235 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:11.709990 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:11.709996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:11.710000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:11.710130 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:11.710545 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:11.710551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:11.710554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:11.710819 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:11.710964 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:11.711612 (+ 648us) server_negotiation.cc:299] Negotiation successful
0810 06:52:11.711821 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":230,"threads_started":1}
I20260810 06:52:11.713078 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:11.695628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:44431 (local address 127.22.11.130:35013)
0810 06:52:11.696486 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:11.696536 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:11.698425 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:11.699169 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:11.699177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:11.699591 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:52:11.700164 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:11.700183 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:11.701261 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:11.701265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:11.702477 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:11.702490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:11.703221 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:11.703228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:11.703441 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:11.704070 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:52:11.704091 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:52:11.706182 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:52:11.709357 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:11.709364 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:11.709378 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:11.709884 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:11.710226 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:11.710231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:11.710235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:11.710430 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:11.710940 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:52:11.710948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:52:11.711248 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 06:52:11.711480 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 06:52:11.711944 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":342,"threads_started":1}
I20260810 06:52:11.714452 10550 heartbeater.cc:347] Connected to a master server at 127.22.11.190:44431
I20260810 06:52:11.714792 10550 heartbeater.cc:464] Registering TS with master...
I20260810 06:52:11.715718 10550 heartbeater.cc:511] Master 127.22.11.190:44431 requested a full tablet report, sending...
I20260810 06:52:11.717444 9923 ts_manager.cc:194] Registered new tserver with Master: 0e61fbb77c084f3682f552a4f5e1754f (127.22.11.130:35995)
I20260810 06:52:11.718513 9923 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:35013
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:52:11.755573 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:59763/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44431
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:52:11.933966 10310 heartbeater.cc:503] Master 127.22.11.190:44431 was elected leader, sending a full tablet report...
W20260810 06:52:12.277247 10563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:12.277494 10563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:12.277593 10563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:12.284540 10563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:12.284654 10563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:12.284771 10563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:52:12.294184 10563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.11.148:59763/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:44431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:12.295320 10563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:12.296499 10563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:12.310592 10572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:12.311076 10575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:12.311115 10573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:12.317050 10563 server_base.cc:1034] running on GCE node
I20260810 06:52:12.317651 10563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:12.318598 10563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:12.319861 10563 hybrid_clock.cc:648] HybridClock initialized: now 1786344732319758 us; error 33 us; skew 500 ppm
I20260810 06:52:12.320242 10563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:12 dist-test-slave-f7th krb5kdc[9081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344732, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:12.324342 10563 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:52:12.326489 10563 webserver.cc:460] Webserver started at http://127.22.11.131:33379/ using document root <none> and password file <none>
I20260810 06:52:12.327313 10563 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:12.327430 10563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:12.327772 10563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:12 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344732, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:12.342769 10563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "fc54762ac902497d8d28c3b2c92096ba"
format_stamp: "Formatted at 2026-08-10 06:52:12 on dist-test-slave-f7th"
server_key: "4200deef1aeaaae912f3aeef08b7f9f4"
server_key_iv: "879ae37487a3b9e096f5a20f078962b0"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:12.343712 10563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "fc54762ac902497d8d28c3b2c92096ba"
format_stamp: "Formatted at 2026-08-10 06:52:12 on dist-test-slave-f7th"
server_key: "4200deef1aeaaae912f3aeef08b7f9f4"
server_key_iv: "879ae37487a3b9e096f5a20f078962b0"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:12.350425 10563 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 06:52:12.369957 10587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:12.371644 10563 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:52:12.371850 10563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "fc54762ac902497d8d28c3b2c92096ba"
format_stamp: "Formatted at 2026-08-10 06:52:12 on dist-test-slave-f7th"
server_key: "4200deef1aeaaae912f3aeef08b7f9f4"
server_key_iv: "879ae37487a3b9e096f5a20f078962b0"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:12.372156 10563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:52:12.407275 10563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:52:12.412825 10563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:52:12.413283 10563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:52:12.414696 10563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:52:12.416658 10563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:52:12.416736 10563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:12.416932 10563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:52:12.416993 10563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:12.489038 10563 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:41619
I20260810 06:52:12.489131 10786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:41619 every 8 connection(s)
I20260810 06:52:12.490545 10563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:52:12.492257 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 10563
I20260810 06:52:12.492426 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:52:12.493124 22574 external_mini_cluster.cc:1428] Setting key 682af4c530c080c338d984c5229dd3de
Aug 10 06:52:12 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344732, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:52:12.515986 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.493828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:36505 (local address 127.22.11.190:44431)
0810 06:52:12.494656 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.494661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.496520 (+ 1859us) server_negotiation.cc:407] Connection header received
0810 06:52:12.499262 (+ 2742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.499268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.499344 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.499434 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:12.501302 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.502294 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.504226 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.504923 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.505042 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.509363 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:12.509368 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:12.509371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:12.509426 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:12.512901 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:12.513652 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:12.513655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:12.513658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:12.513833 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:12.514309 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:12.514312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:12.514314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:12.514682 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:12.514792 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.515400 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.515571 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":702,"thread_start_us":506,"threads_started":1}
I20260810 06:52:12.517160 10791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.494711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:44431 (local address 127.22.11.131:36505)
0810 06:52:12.496368 (+ 1657us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:12.496421 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:12.499008 (+ 2587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:12.499725 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:12.499734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:12.500264 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:52:12.501142 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.501161 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.502429 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.502433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.504084 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.504097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.505145 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.505153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.505438 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.506106 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:52:12.506128 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:52:12.509096 (+ 2968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:52:12.513018 (+ 3922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:12.513025 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:12.513035 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:12.513559 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:12.513915 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:12.513918 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:12.513920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:12.514225 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:12.514781 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:52:12.514786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:52:12.515083 (+ 297us) client_negotiation.cc:769] Sending connection context
0810 06:52:12.515331 (+ 248us) client_negotiation.cc:240] Negotiation successful
0810 06:52:12.515786 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"thread_start_us":879,"threads_started":1}
I20260810 06:52:12.518409 10787 heartbeater.cc:347] Connected to a master server at 127.22.11.190:44431
I20260810 06:52:12.518741 10787 heartbeater.cc:464] Registering TS with master...
I20260810 06:52:12.519685 10787 heartbeater.cc:511] Master 127.22.11.190:44431 requested a full tablet report, sending...
I20260810 06:52:12.521744 9922 ts_manager.cc:194] Registered new tserver with Master: fc54762ac902497d8d28c3b2c92096ba (127.22.11.131:41619)
I20260810 06:52:12.523087 9922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:36505
I20260810 06:52:12.534279 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:52:12.556831 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.540124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49638 (local address 127.22.11.190:44431)
0810 06:52:12.540265 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.540269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.541223 (+ 954us) server_negotiation.cc:407] Connection header received
0810 06:52:12.541418 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.541422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.541528 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.541614 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:12.544005 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.545020 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.546821 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.547399 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.547556 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.550407 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:12.550412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:12.550416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:12.550447 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:12.553841 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:12.554601 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:12.554607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:12.554610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:12.554736 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:12.555149 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:12.555154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:12.555157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:12.555492 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:12.555783 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.556153 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.556392 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:52:12.561055 9922 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:52:12.571660 9922 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:52:12.600284 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.586729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55414 (local address 127.22.11.130:35995)
0810 06:52:12.587482 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.587488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.588528 (+ 1040us) server_negotiation.cc:407] Connection header received
0810 06:52:12.588866 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.588874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.589126 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.589326 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:52:12.591054 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.593213 (+ 2159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.597270 (+ 4057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.598855 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.598986 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.599282 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.599677 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.599842 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":515,"threads_started":1}
I20260810 06:52:12.600483 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.587095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35995 (local address 127.0.0.1:55414)
0810 06:52:12.588384 (+ 1289us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:12.588417 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:12.588600 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:12.589816 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:12.589824 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:12.589855 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:52:12.590787 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.590803 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.593384 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.593390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.596560 (+ 3170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.596578 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.599069 (+ 2491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.599076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.599413 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.599475 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 06:52:12.599619 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 06:52:12.599826 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":190,"thread_start_us":811,"threads_started":1}
I20260810 06:52:12.603683 10819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.589118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.22.11.131:41619)
0810 06:52:12.590872 (+ 1754us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.590881 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.591690 (+ 809us) server_negotiation.cc:407] Connection header received
0810 06:52:12.591772 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.591777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.591930 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.592082 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:52:12.594618 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.596296 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.599989 (+ 3693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.601863 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.601939 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.602177 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.602966 (+ 789us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.603165 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1512,"thread_start_us":1207,"threads_started":1}
I20260810 06:52:12.606017 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.588726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:41619 (local address 127.0.0.1:34674)
0810 06:52:12.591453 (+ 2727us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:12.591467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:12.591582 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:12.592085 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:12.592089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:12.592111 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:52:12.594386 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.594404 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.596467 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.596473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.599810 (+ 3337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.599824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.602460 (+ 2636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.602468 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.602734 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.602779 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 06:52:12.602875 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:52:12.603005 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2599,"mutex_wait_us":3,"thread_start_us":2506,"threads_started":1}
I20260810 06:52:12.605397 10442 tablet_service.cc:1467] Processing CreateTablet for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 (DEFAULT_TABLE table=test-table [id=e3f851ac90b54b3f86b199e7d97f9a21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:52:12.607156 10442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:52:12.608803 10827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.591565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36031 (local address 127.0.0.1:54536)
0810 06:52:12.598512 (+ 6947us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:12.598524 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:12.598643 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:12.599614 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:12.599618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:12.599644 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:52:12.600029 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.600050 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.602528 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.602533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.605720 (+ 3187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.605735 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.607683 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.607690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.607927 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.607959 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:52:12.608056 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:52:12.608171 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6837,"thread_start_us":6657,"threads_started":1}
I20260810 06:52:12.609109 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.592011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.22.11.129:36031)
0810 06:52:12.597691 (+ 5680us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.597700 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.598627 (+ 927us) server_negotiation.cc:407] Connection header received
0810 06:52:12.598809 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.598815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.598983 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.599215 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:52:12.600251 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.602376 (+ 2125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.605897 (+ 3521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.607320 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.607452 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.607737 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.608164 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.608364 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5495,"thread_start_us":5294,"threads_started":1}
I20260810 06:52:12.610483 10667 tablet_service.cc:1467] Processing CreateTablet for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 (DEFAULT_TABLE table=test-table [id=e3f851ac90b54b3f86b199e7d97f9a21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:52:12.612342 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:52:12.613931 10184 tablet_service.cc:1467] Processing CreateTablet for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 (DEFAULT_TABLE table=test-table [id=e3f851ac90b54b3f86b199e7d97f9a21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:52:12.615761 10184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:52:12.622448 10828 tablet_bootstrap.cc:492] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Bootstrap starting.
I20260810 06:52:12.627468 10828 tablet_bootstrap.cc:654] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Neither blocks nor log segments found. Creating new log.
I20260810 06:52:12.629333 10828 log.cc:824] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:52:12.630476 10830 tablet_bootstrap.cc:492] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Bootstrap starting.
I20260810 06:52:12.634102 10833 tablet_bootstrap.cc:492] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Bootstrap starting.
I20260810 06:52:12.634979 10830 tablet_bootstrap.cc:654] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Neither blocks nor log segments found. Creating new log.
I20260810 06:52:12.635249 10828 tablet_bootstrap.cc:492] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: No bootstrap required, opened a new log
I20260810 06:52:12.635542 10828 ts_tablet_manager.cc:1397] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260810 06:52:12.636444 10830 log.cc:824] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Log is configured to *not* fsync() on all Append() calls
I20260810 06:52:12.638769 10833 tablet_bootstrap.cc:654] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Neither blocks nor log segments found. Creating new log.
I20260810 06:52:12.640090 10833 log.cc:824] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Log is configured to *not* fsync() on all Append() calls
I20260810 06:52:12.640252 10830 tablet_bootstrap.cc:492] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: No bootstrap required, opened a new log
I20260810 06:52:12.640538 10830 ts_tablet_manager.cc:1397] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 06:52:12.647305 10833 tablet_bootstrap.cc:492] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: No bootstrap required, opened a new log
I20260810 06:52:12.647604 10833 ts_tablet_manager.cc:1397] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260810 06:52:12.649690 10828 raft_consensus.cc:348] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.650600 10828 raft_consensus.cc:374] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:52:12.650775 10828 raft_consensus.cc:729] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e61fbb77c084f3682f552a4f5e1754f, State: Initialized, Role: FOLLOWER
I20260810 06:52:12.651623 10828 consensus_queue.cc:260] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.653551 10550 heartbeater.cc:503] Master 127.22.11.190:44431 was elected leader, sending a full tablet report...
I20260810 06:52:12.654301 10828 ts_tablet_manager.cc:1428] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Time spent starting tablet: real 0.019s user 0.012s sys 0.006s
I20260810 06:52:12.656424 10830 raft_consensus.cc:348] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.657037 10830 raft_consensus.cc:374] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:52:12.657188 10830 raft_consensus.cc:729] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc54762ac902497d8d28c3b2c92096ba, State: Initialized, Role: FOLLOWER
I20260810 06:52:12.657895 10830 consensus_queue.cc:260] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.659623 10787 heartbeater.cc:503] Master 127.22.11.190:44431 was elected leader, sending a full tablet report...
I20260810 06:52:12.660398 10830 ts_tablet_manager.cc:1428] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Time spent starting tablet: real 0.020s user 0.013s sys 0.005s
I20260810 06:52:12.660795 10833 raft_consensus.cc:348] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.661597 10833 raft_consensus.cc:374] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:52:12.661758 10833 raft_consensus.cc:729] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cbeef5980dc4de18d0a3789e42ed307, State: Initialized, Role: FOLLOWER
I20260810 06:52:12.662398 10833 consensus_queue.cc:260] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.664685 10833 ts_tablet_manager.cc:1428] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
W20260810 06:52:12.694442 10551 tablet.cc:2369] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:52:12.724741 10842 raft_consensus.cc:482] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:52:12.725195 10842 raft_consensus.cc:504] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.727425 10842 leader_election.cc:288] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e61fbb77c084f3682f552a4f5e1754f (127.22.11.130:35995), fc54762ac902497d8d28c3b2c92096ba (127.22.11.131:41619)
I20260810 06:52:12.736225 10819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.729585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:50879 (local address 127.22.11.131:41619)
0810 06:52:12.729827 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.729832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.730164 (+ 332us) server_negotiation.cc:407] Connection header received
0810 06:52:12.730339 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.730344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.730421 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.730541 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:52:12.731506 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.732722 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.734465 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.735336 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.735452 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.735549 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.735865 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.735991 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:52:12.736317 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.729091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:41619 (local address 127.22.11.129:50879)
0810 06:52:12.730027 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:12.730073 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:12.730210 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:12.730554 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:12.730557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:12.730583 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:52:12.731310 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.731321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.732844 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.732848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.734307 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.734321 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.735545 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.735551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.735710 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.735746 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:52:12.735827 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:52:12.735918 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":2,"thread_start_us":589,"threads_started":1}
I20260810 06:52:12.736544 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.727777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:39227 (local address 127.22.11.130:35995)
0810 06:52:12.727983 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.727988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.729222 (+ 1234us) server_negotiation.cc:407] Connection header received
0810 06:52:12.729519 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.729524 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.729602 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.729669 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:52:12.730837 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.732054 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.734247 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.735531 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.735675 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.735742 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.736191 (+ 449us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.736311 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:52:12.736728 10843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.727947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35995 (local address 127.22.11.129:39227)
0810 06:52:12.729101 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:12.729120 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:12.729320 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:12.729937 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:12.729940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:12.729963 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:52:12.730692 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.730709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.732196 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.732202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.734058 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:12.734071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.735764 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.735771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:12.736000 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.736033 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:52:12.736130 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:52:12.736248 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"mutex_wait_us":35,"thread_start_us":678,"threads_started":1}
I20260810 06:52:12.737571 10702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb44b8f6d348c0b30fbf8fac6a1e70" candidate_uuid: "2cbeef5980dc4de18d0a3789e42ed307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc54762ac902497d8d28c3b2c92096ba" is_pre_election: true
I20260810 06:52:12.737582 10477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb44b8f6d348c0b30fbf8fac6a1e70" candidate_uuid: "2cbeef5980dc4de18d0a3789e42ed307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e61fbb77c084f3682f552a4f5e1754f" is_pre_election: true
I20260810 06:52:12.737999 10477 raft_consensus.cc:2454] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cbeef5980dc4de18d0a3789e42ed307 in term 0.
I20260810 06:52:12.738018 10702 raft_consensus.cc:2454] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cbeef5980dc4de18d0a3789e42ed307 in term 0.
I20260810 06:52:12.738651 10104 leader_election.cc:302] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cbeef5980dc4de18d0a3789e42ed307, fc54762ac902497d8d28c3b2c92096ba; no voters:
I20260810 06:52:12.738982 10842 raft_consensus.cc:2781] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:52:12.739135 10842 raft_consensus.cc:482] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:52:12.739236 10842 raft_consensus.cc:3037] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:52:12.741453 10842 raft_consensus.cc:504] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.742012 10842 leader_election.cc:288] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [CANDIDATE]: Term 1 election: Requested vote from peers 0e61fbb77c084f3682f552a4f5e1754f (127.22.11.130:35995), fc54762ac902497d8d28c3b2c92096ba (127.22.11.131:41619)
I20260810 06:52:12.742370 10477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb44b8f6d348c0b30fbf8fac6a1e70" candidate_uuid: "2cbeef5980dc4de18d0a3789e42ed307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e61fbb77c084f3682f552a4f5e1754f"
I20260810 06:52:12.742538 10477 raft_consensus.cc:3037] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:52:12.742468 10702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3cb44b8f6d348c0b30fbf8fac6a1e70" candidate_uuid: "2cbeef5980dc4de18d0a3789e42ed307" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc54762ac902497d8d28c3b2c92096ba"
I20260810 06:52:12.742697 10702 raft_consensus.cc:3037] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 0 FOLLOWER]: Advancing to term 1
W20260810 06:52:12.743364 10788 tablet.cc:2369] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:52:12.744921 10477 raft_consensus.cc:2454] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cbeef5980dc4de18d0a3789e42ed307 in term 1.
I20260810 06:52:12.745275 10104 leader_election.cc:302] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e61fbb77c084f3682f552a4f5e1754f, 2cbeef5980dc4de18d0a3789e42ed307; no voters:
I20260810 06:52:12.745345 10702 raft_consensus.cc:2454] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cbeef5980dc4de18d0a3789e42ed307 in term 1.
I20260810 06:52:12.745537 10842 raft_consensus.cc:2781] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:52:12.745981 10842 raft_consensus.cc:686] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 1 LEADER]: Becoming Leader. State: Replica: 2cbeef5980dc4de18d0a3789e42ed307, State: Running, Role: LEADER
I20260810 06:52:12.746403 10842 consensus_queue.cc:237] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } }
I20260810 06:52:12.750485 9922 catalog_manager.cc:5495] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cbeef5980dc4de18d0a3789e42ed307 (127.22.11.129). New cstate: current_term: 1 leader_uuid: "2cbeef5980dc4de18d0a3789e42ed307" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cbeef5980dc4de18d0a3789e42ed307" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36031 } health_report { overall_health: HEALTHY } } }
I20260810 06:52:12.787119 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.782453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.22.11.129:36031)
0810 06:52:12.782626 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.782630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.782647 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:52:12.782755 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.782758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.782835 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.782957 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:52:12.783270 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.783979 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.785339 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.785716 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.785802 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.786200 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:52:12.786674 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:52:12.786776 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.786858 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.786918 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:52:12.794746 10477 raft_consensus.cc:1261] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 1 FOLLOWER]: Refusing update from remote peer 2cbeef5980dc4de18d0a3789e42ed307: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:52:12.794768 10702 raft_consensus.cc:1261] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 1 FOLLOWER]: Refusing update from remote peer 2cbeef5980dc4de18d0a3789e42ed307: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:52:12.795629 10849 consensus_queue.cc:1035] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e61fbb77c084f3682f552a4f5e1754f" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:52:12.795964 10842 consensus_queue.cc:1035] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc54762ac902497d8d28c3b2c92096ba" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:52:12.805624 10856 mvcc.cc:205] Tried to move back new op lower bound from 7316868025517944832 to 7316868025332568064. Current Snapshot: MvccSnapshot[applied={T|T < 7316868025517944832}]
I20260810 06:52:12.807925 10858 mvcc.cc:205] Tried to move back new op lower bound from 7316868025517944832 to 7316868025332568064. Current Snapshot: MvccSnapshot[applied={T|T < 7316868025517944832}]
I20260810 06:52:12.809738 10859 mvcc.cc:205] Tried to move back new op lower bound from 7316868025517944832 to 7316868025332568064. Current Snapshot: MvccSnapshot[applied={T|T < 7316868025517944832}]
I20260810 06:52:12.820999 9922 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:52:12.821310 9922 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:52:12.825074 9922 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 455ac090eb7c4415ba2dbb105e55b1ee: Sending DeleteTablet for 3 replicas of tablet d3cb44b8f6d348c0b30fbf8fac6a1e70
I20260810 06:52:12.826094 10442 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:52:12 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:55414
I20260810 06:52:12.826479 10184 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:52:12 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:54536
I20260810 06:52:12.827355 10667 tablet_service.cc:1514] Processing DeleteTablet for tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:52:12 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:34674
I20260810 06:52:12.827860 10873 tablet_replica.cc:330] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: stopping tablet replica
I20260810 06:52:12.828241 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 10055
I20260810 06:52:12.828289 10873 raft_consensus.cc:2229] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:52:12.829001 10873 raft_consensus.cc:2258] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:52:12.829711 10871 tablet_replica.cc:330] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: stopping tablet replica
I20260810 06:52:12.830016 10871 raft_consensus.cc:2229] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:52:12.830340 10871 raft_consensus.cc:2258] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:52:12.831099 10874 tablet_replica.cc:330] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: stopping tablet replica
I20260810 06:52:12.831450 10874 raft_consensus.cc:2229] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:52:12.831938 10874 raft_consensus.cc:2258] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:52:12.832393 10873 ts_tablet_manager.cc:1905] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:52:12.834383 10871 ts_tablet_manager.cc:1905] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:52:12.839989 10873 ts_tablet_manager.cc:1918] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:52:12.840248 10873 log.cc:1196] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/d3cb44b8f6d348c0b30fbf8fac6a1e70
I20260810 06:52:12.840850 10873 ts_tablet_manager.cc:1939] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 2cbeef5980dc4de18d0a3789e42ed307: Deleting consensus metadata
I20260810 06:52:12.841224 10871 ts_tablet_manager.cc:1918] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:52:12.841418 10871 log.cc:1196] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/d3cb44b8f6d348c0b30fbf8fac6a1e70
I20260810 06:52:12.841955 10871 ts_tablet_manager.cc:1939] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P 0e61fbb77c084f3682f552a4f5e1754f: Deleting consensus metadata
I20260810 06:52:12.842000 10874 ts_tablet_manager.cc:1905] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:52:12.842833 9896 catalog_manager.cc:4841] TS 2cbeef5980dc4de18d0a3789e42ed307 (127.22.11.129:36031): tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 (table test-table [id=e3f851ac90b54b3f86b199e7d97f9a21]) successfully deleted
I20260810 06:52:12.843641 9896 catalog_manager.cc:4841] TS 0e61fbb77c084f3682f552a4f5e1754f (127.22.11.130:35995): tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 (table test-table [id=e3f851ac90b54b3f86b199e7d97f9a21]) successfully deleted
I20260810 06:52:12.850342 10874 ts_tablet_manager.cc:1918] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:52:12.850521 10874 log.cc:1196] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/d3cb44b8f6d348c0b30fbf8fac6a1e70
I20260810 06:52:12.850996 10874 ts_tablet_manager.cc:1939] T d3cb44b8f6d348c0b30fbf8fac6a1e70 P fc54762ac902497d8d28c3b2c92096ba: Deleting consensus metadata
I20260810 06:52:12.852419 9896 catalog_manager.cc:4841] TS fc54762ac902497d8d28c3b2c92096ba (127.22.11.131:41619): tablet d3cb44b8f6d348c0b30fbf8fac6a1e70 (table test-table [id=e3f851ac90b54b3f86b199e7d97f9a21]) successfully deleted
Aug 10 06:52:12 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:52:12.861478 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.829906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54564 (local address 127.22.11.129:36031)
0810 06:52:12.830114 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.830120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.843923 (+ 13803us) server_negotiation.cc:407] Connection header received
0810 06:52:12.844135 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.844140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.844214 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.844285 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:12.846209 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.847399 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.849380 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.850085 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.850222 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.855428 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:12.855440 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:12.855455 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:12.855496 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:12.858500 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:12.859336 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:12.859343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:12.859345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:12.859500 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:12.859893 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:12.859899 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:12.859901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:12.860270 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:12.860563 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:12.860893 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 06:52:12.861099 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:52:12.862141 10297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:52:12.962890 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 10055
I20260810 06:52:12.987140 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 10318
Aug 10 06:52:12 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:52:13.004285 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:12.987958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.22.11.130:35995)
0810 06:52:12.988119 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:12.988124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:12.988143 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:52:12.988252 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:12.988256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:12.988319 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:12.988440 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:12.990031 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.990835 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.992810 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:12.993268 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:12.993383 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:12.998191 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:12.998206 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:12.998222 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:12.998259 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:13.001248 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:13.001910 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:13.001918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:13.001920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:13.002052 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:13.002435 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:13.002439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:13.002441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:13.002908 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:13.003347 (+ 439us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:13.003542 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 06:52:13.003749 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:52:13.005095 10542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:52:13.099718 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 10318
I20260810 06:52:13.120358 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 10563
Aug 10 06:52:13 dist-test-slave-f7th krb5kdc[9081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:52:13.134825 10819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:13.121202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.22.11.131:41619)
0810 06:52:13.121373 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:13.121377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:13.121395 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:52:13.121451 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:13.121455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:13.121530 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:13.121621 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:13.122993 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:13.123711 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:13.125003 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:13.125396 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:13.125488 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:13.129692 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:13.129708 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:13.129726 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:13.129761 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:13.132486 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:13.133110 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:13.133118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:13.133120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:13.133239 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:13.133563 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:13.133566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:13.133568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:13.133927 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:13.134165 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:13.134385 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 06:52:13.134552 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:52:13.135455 10778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:52:13.223119 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 10563
I20260810 06:52:13.247044 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 9865
I20260810 06:52:13.248391 9975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:52:13.360744 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 9865
I20260810 06:52:13.394374 22574 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:52:13.738353 22574 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:52:13.738639 22574 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:52:14.083598 22574 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:52:14.083 UTC [9141] LOG: received smart shutdown request
2026-08-10 06:52:14.087 UTC [9141] LOG: background worker "logical replication launcher" (PID 9152) exited with exit code 1
2026-08-10 06:52:14.087 UTC [9147] LOG: shutting down
2026-08-10 06:52:14.212 UTC [9141] LOG: database system is shut down
2026-08-10T06:52:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49348 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:52:14 dist-test-slave-f7th krb5kdc[10902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:52:14 dist-test-slave-f7th krb5kdc[10902](info): set up 2 sockets
Aug 10 06:52:14 dist-test-slave-f7th krb5kdc[10902](info): commencing operation
krb5kdc: starting...
W20260810 06:52:15.567829 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.002s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:52:15 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:52:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/rangeradmin_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/rangeradmin_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/rangerlookup_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/rangerlookup_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
I20260810 06:52:15.899065 22574 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskJIbnRf/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/postgres -l logfile start
2026-08-10 06:52:18.065 UTC [10960] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 06:52:18.065 UTC [10960] LOG: listening on IPv4 address "127.22.11.148", port 49219
2026-08-10 06:52:18.068 UTC [10960] LOG: listening on Unix socket "/tmp/.s.PGSQL.49219"
2026-08-10 06:52:18.088 UTC [10964] LOG: database system was shut down at 2026-08-10 06:52:16 UTC
2026-08-10 06:52:18.091 UTC [10960] LOG: database system is ready to accept connections
I20260810 06:52:19.210531 22574 mini_postgres.cc:96] Postgres bound to 49219
2026-08-10 06:52:19.232 UTC [10980] FATAL: database "slave" does not exist
127.22.11.148:49219 - accepting connections
I20260810 06:52:19.233179 22574 mini_ranger.cc:161] Starting Ranger...
I20260810 06:52:19.262853 22574 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 06:52:19.684852 22574 mini_ranger.cc:87] Created ranger Postgres database
I20260810 06:52:19.684963 22574 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-admin
2026-08-10 06:52:19,991 [I] DB FLAVOR :POSTGRES
2026-08-10 06:52:19,991 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:52:19,991 [I] Checking connection..
2026-08-10 06:52:19,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 06:52:20,226 [I] Checking connection passed.
2026-08-10 06:52:20,227 [I] --------- Verifying version history table ---------
2026-08-10 06:52:20,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:52:20,488 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:52:20,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:52:20,749 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 06:52:20,750 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 06:52:20,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 06:52:20.992 UTC [11103] WARNING: there is no transaction in progress
2026-08-10 06:52:21,003 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 06:52:21,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 06:52:21,238 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 06:52:21,238 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:52:21,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 06:52:21,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 06:52:21,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-f7th.c.gcp-upstream.internal','N') ;"
2026-08-10 06:52:21,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:52:22,220 [I] Table x_portal_user does not exist in database ranger
2026-08-10 06:52:22,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:52:22,460 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 06:52:22,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:52:22,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:52:22,960 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 06:52:22,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 06:52:23.863 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:23.879 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:23.893 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:23.907 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:23.921 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:23.937 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:23.988 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:23.994 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.006 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.016 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.025 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.038 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.047 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.052 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.058 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24.064 UTC [11308] WARNING: there is no transaction in progress
2026-08-10 06:52:24,628 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 06:52:24,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 06:52:24,867 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 06:52:24,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 06:52:25,114 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 06:52:25,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 06:52:25,364 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 06:52:25,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 06:52:25,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 06:52:25,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-f7th.c.gcp-upstream.internal';"
2026-08-10 06:52:26,104 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 06:52:26,104 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 06:52:26,358 [I] DB_PATCHES have already been applied
I20260810 06:52:26.363982 22574 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-admin:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskJIbnRf/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 06:52:26.364078 22574 mini_ranger.cc:193] Using host: 127.22.11.148
I20260810 06:52:26.382911 22574 mini_ranger.cc:227] Ranger admin URL: http://127.22.11.148:44635
Aug 10, 2026 6:52:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:52:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:52:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 6:52:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 6:52:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 6:52:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 6:52:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 06:52:27.633867 22574 mini_ranger.cc:160] Time spent starting Ranger: real 8.401s user 0.004s sys 0.049s
Aug 10, 2026 6:52:46 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 6:52:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 6:52:46 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 6:52:46 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 6:52:47 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 06:52:47.912577 22574 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.22.11.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.22.11.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/rangerkms_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.22.11.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/rangerkms_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.22.11.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.22.11.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.22.11.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/HTTP_127.22.11.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 06:52:48 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344768, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 06:52:48.091712 22574 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 06:52:48.113371 22574 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 06:52:48.803045 22574 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 06:52:48.803179 22574 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms
I20260810 06:52:48.803229 22574 mini_ranger_kms.cc:227] Using postgres at 127.22.11.148:49219
2026-08-10 06:52:49,156 [I] DB FLAVOR :POSTGRES
2026-08-10 06:52:49,157 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 06:52:49,157 [I] Checking connection
2026-08-10 06:52:49,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 06:52:49,395 [I] connection success
2026-08-10 06:52:49,395 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 06:52:49,395 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 06:52:49,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 06:52:49,621 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 06:52:49,622 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 06:52:49,622 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 06:52:49,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.22.11.148:49219/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 06:52:49,865 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 06:52:50.346789 22574 mini_ranger_kms.cc:310] Created kms service
I20260810 06:52:50.529682 22574 mini_ranger_kms.cc:326] Created kudu user
I20260810 06:52:50.568595 22574 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 06:52:50 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344768, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:50.956354 22574 mini_ranger_kms.cc:384] Added ranger policy
I20260810 06:52:50.956710 22574 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms:/tmp/dist-test-taskJIbnRf/build/asan/bin/postgresql.jar:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskJIbnRf/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 06:52:50.956776 22574 mini_ranger_kms.cc:240] Using host: 127.22.11.148
I20260810 06:52:50.981314 22574 mini_ranger_kms.cc:276] Ranger KMS PID: 11659
I20260810 06:52:50.981436 22574 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.22.11.148:56065
Aug 10, 2026 6:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 6:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 6:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 6:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 6:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 6:52:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 06:52:53 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.11.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:53.710618 22574 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.619s user 0.048s sys 0.336s
I20260810 06:52:53.710871 22574 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 06:52:53.711061 22574 mini_ranger_kms.cc:401] 127.22.11.148:56065/kms/v1/keys
Aug 10 06:52:53 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.11.148@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
Aug 10 06:52:54 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344773, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.22.11.148@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
Aug 10 06:52:56 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:52:56.582159 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34173
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34173
--ranger_config_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:52:56.919190 11753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:56.919461 11753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:56.919557 11753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:56.926512 11753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:52:56.926602 11753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:56.926653 11753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:56.926709 11753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:52:56.926756 11753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:52:56.936501 11753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
--ranger_kms_url=127.22.11.148:56065/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34173
--ranger_config_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:56.937721 11753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:56.939009 11753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:56.950029 11762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:56.950076 11763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:56.950115 11765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:56.951107 11753 server_base.cc:1034] running on GCE node
I20260810 06:52:56.951802 11753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:56.953146 11753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:56.954332 11753 hybrid_clock.cc:648] HybridClock initialized: now 1786344776954294 us; error 33 us; skew 500 ppm
I20260810 06:52:56.954641 11753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:56 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344776, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:56.960767 11753 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:52:56.962157 11753 webserver.cc:460] Webserver started at http://127.22.11.190:42521/ using document root <none> and password file <none>
I20260810 06:52:56.962729 11753 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:56.962824 11753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:56.963093 11753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:56 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344776, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:57.027518 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "7d632c00baec4cf4b9f3b90095e1dcf8"
format_stamp: "Formatted at 2026-08-10 06:52:57 on dist-test-slave-f7th"
server_key: "3192b03351ddc67ab8ed650dfd6bb450"
server_key_iv: "3b846ca0674056f63c9a46f9a4d44539"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:57.028355 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "7d632c00baec4cf4b9f3b90095e1dcf8"
format_stamp: "Formatted at 2026-08-10 06:52:57 on dist-test-slave-f7th"
server_key: "3192b03351ddc67ab8ed650dfd6bb450"
server_key_iv: "3b846ca0674056f63c9a46f9a4d44539"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:57.033607 11753 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:52:57.051836 11782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:57.053501 11753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:52:57.053695 11753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "7d632c00baec4cf4b9f3b90095e1dcf8"
format_stamp: "Formatted at 2026-08-10 06:52:57 on dist-test-slave-f7th"
server_key: "3192b03351ddc67ab8ed650dfd6bb450"
server_key_iv: "3b846ca0674056f63c9a46f9a4d44539"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:57.053936 11753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:52:57.084105 11753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:52:57.088904 11753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:52:57.089250 11753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:52:57.108114 11886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34173 every 8 connection(s)
I20260810 06:52:57.108141 11753 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34173
I20260810 06:52:57.109880 11753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:52:57.113737 11888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:52:57.119057 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 11753
I20260810 06:52:57.119280 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:52:57.120127 22574 external_mini_cluster.cc:1428] Setting key 1bb89a197bf7ec5092c74f27d7419e7a
I20260810 06:52:57.125159 11888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Bootstrap starting.
I20260810 06:52:57.129347 11888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Neither blocks nor log segments found. Creating new log.
I20260810 06:52:57.130452 11888 log.cc:824] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:52:57.133167 11888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: No bootstrap required, opened a new log
Aug 10 06:52:57 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:52:57.140511 11888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:52:57.140885 11888 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:52:57.140985 11888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d632c00baec4cf4b9f3b90095e1dcf8, State: Initialized, Role: FOLLOWER
I20260810 06:52:57.141562 11888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:52:57.141834 11888 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:52:57.141961 11888 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:52:57.142141 11888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:52:57.144423 11888 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:52:57.145008 11888 leader_election.cc:302] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d632c00baec4cf4b9f3b90095e1dcf8; no voters:
I20260810 06:52:57.145635 11888 leader_election.cc:288] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:52:57.145761 11898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:52:57.146826 11898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 LEADER]: Becoming Leader. State: Replica: 7d632c00baec4cf4b9f3b90095e1dcf8, State: Running, Role: LEADER
I20260810 06:52:57.147475 11898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:52:57.148665 11894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:57.122364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47888 (local address 127.22.11.190:34173)
0810 06:52:57.129197 (+ 6833us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:57.129214 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:57.129573 (+ 359us) server_negotiation.cc:407] Connection header received
0810 06:52:57.131485 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:57.131516 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:57.131860 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:57.132183 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:57.133588 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:57.134893 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:57.136155 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:57.136877 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:57.137057 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:57.140903 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:57.140912 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:57.140927 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:57.140972 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:57.143977 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:57.144814 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:57.144823 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:57.144831 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:57.145061 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:57.145471 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:57.145478 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:57.145482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:57.146296 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:57.146585 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:57.147081 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 06:52:57.147475 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6593,"thread_start_us":6359,"threads_started":1}
I20260810 06:52:57.147730 11888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:52:57.150120 11900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } } }
I20260810 06:52:57.150465 11900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:52:57.153218 11901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d632c00baec4cf4b9f3b90095e1dcf8. Latest consensus state: current_term: 1 leader_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } } }
I20260810 06:52:57.153525 11901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:52:57.154805 11906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:52:57.159724 11906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:52:57.160555 11888 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:52:57.179023 11906 catalog_manager.cc:1347] Generated new cluster ID: 8be0e805745841d7b9a3dfbcf602e219
I20260810 06:52:57.179162 11906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:52:57.190838 11906 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:52:57.192703 11906 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:52:57.211036 11906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Generated new TSK 0
I20260810 06:52:57.212086 11906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:52:57.965426 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34173
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:52:58 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344778, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 06:52:58.397137 11958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:58.397436 11958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:58.397588 11958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:58.404722 11958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:58.404937 11958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:58.405148 11958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:52:58.417062 11958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.22.11.148:56065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:58.418349 11958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:58.419651 11958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:58.435837 11980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:58.437034 11981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:58.454640 11958 server_base.cc:1034] running on GCE node
W20260810 06:52:58.454937 11983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:58.455725 11958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:58.456831 11958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:58.458123 11958 hybrid_clock.cc:648] HybridClock initialized: now 1786344778458057 us; error 48 us; skew 500 ppm
I20260810 06:52:58.458482 11958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:58 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344778, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:58.461796 11958 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:52:58.464957 11958 webserver.cc:460] Webserver started at http://127.22.11.129:34719/ using document root <none> and password file <none>
I20260810 06:52:58.465669 11958 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:58.465786 11958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:58.466109 11958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:58 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344778, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:58.493700 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "bfb0b6231447409d9f06363dda0bc688"
format_stamp: "Formatted at 2026-08-10 06:52:58 on dist-test-slave-f7th"
server_key: "c786369cb73e4eba84613e0c7f40772e"
server_key_iv: "bbe0ca546a4bf3e6f4d1b1d7ff0d0c5e"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:58.494880 11958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "bfb0b6231447409d9f06363dda0bc688"
format_stamp: "Formatted at 2026-08-10 06:52:58 on dist-test-slave-f7th"
server_key: "c786369cb73e4eba84613e0c7f40772e"
server_key_iv: "bbe0ca546a4bf3e6f4d1b1d7ff0d0c5e"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:58.505586 11958 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.000s sys 0.006s
I20260810 06:52:58.529300 11998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:58.531270 11958 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.004s
I20260810 06:52:58.531531 11958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "bfb0b6231447409d9f06363dda0bc688"
format_stamp: "Formatted at 2026-08-10 06:52:58 on dist-test-slave-f7th"
server_key: "c786369cb73e4eba84613e0c7f40772e"
server_key_iv: "bbe0ca546a4bf3e6f4d1b1d7ff0d0c5e"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:58.531798 11958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:52:58.558713 11958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:52:58.564055 11958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:52:58.564476 11958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:52:58.569000 11958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:52:58.573475 11958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:52:58.573626 11958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:58.573843 11958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:52:58.573988 11958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:58.704794 11958 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:43479
I20260810 06:52:58.704825 12193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:43479 every 8 connection(s)
I20260810 06:52:58.706259 11958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:52:58.716252 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 11958
I20260810 06:52:58.716490 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:52:58.717365 22574 external_mini_cluster.cc:1428] Setting key edac1cb69d146490ae4b1426556a5d04
Aug 10 06:52:58 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344778, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:52:58.752907 12200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:58.709191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:33489 (local address 127.22.11.190:34173)
0810 06:52:58.720001 (+ 10810us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:58.720011 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:58.731202 (+ 11191us) server_negotiation.cc:407] Connection header received
0810 06:52:58.731285 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:58.731289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:58.731386 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:58.731487 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:58.733630 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:58.734758 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:58.739703 (+ 4945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:58.740300 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:58.740411 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:58.744962 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:58.744969 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:58.744974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:58.745013 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:58.748462 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:58.749466 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:58.749472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:58.749475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:58.749758 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:58.750297 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:58.750302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:58.750305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:58.750667 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:58.750825 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:58.752232 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0810 06:52:58.752463 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10647,"thread_start_us":10810,"threads_started":1}
I20260810 06:52:58.754000 12204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:58.709838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34173 (local address 127.22.11.129:33489)
0810 06:52:58.728453 (+ 18615us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:58.728513 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:58.730822 (+ 2309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:58.731951 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:58.731964 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:58.732587 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:52:58.733426 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:58.733451 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:58.737528 (+ 4077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:58.737534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:58.739523 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:58.739537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:58.740717 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:58.740723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:58.740989 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:58.741974 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:52:58.742005 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:52:58.744691 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:52:58.748639 (+ 3948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:58.748648 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:58.748665 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:58.749316 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:58.749884 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:58.749888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:58.749891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:58.750163 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:58.750911 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:52:58.750918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:52:58.751320 (+ 402us) client_negotiation.cc:769] Sending connection context
0810 06:52:58.751613 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 06:52:58.752276 (+ 663us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18281,"thread_start_us":18044,"threads_started":1}
I20260810 06:52:58.756335 12194 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34173
I20260810 06:52:58.756700 12194 heartbeater.cc:464] Registering TS with master...
I20260810 06:52:58.757632 12194 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
I20260810 06:52:58.760257 11814 ts_manager.cc:194] Registered new tserver with Master: bfb0b6231447409d9f06363dda0bc688 (127.22.11.129:43479)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Aug 10 06:52:58 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344778, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:58.764791 11814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:33489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:52:58.805649 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34173
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:52:59 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344778, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
W20260810 06:52:59.301514 12214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:59.301987 12214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:59.302218 12214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:59.313706 12214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:59.313846 12214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:59.314040 12214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:52:59.329286 12214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.22.11.148:56065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:59.330976 12214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:59.332777 12214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:59.353140 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:59.353614 12227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:59.358964 12214 server_base.cc:1034] running on GCE node
W20260810 06:52:59.359373 12231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:59.360208 12214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:59.371712 12214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:59.372922 12214 hybrid_clock.cc:648] HybridClock initialized: now 1786344779372812 us; error 106 us; skew 500 ppm
I20260810 06:52:59.373323 12214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:59 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344779, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:59.377826 12214 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:52:59.379956 12214 webserver.cc:460] Webserver started at http://127.22.11.130:41171/ using document root <none> and password file <none>
I20260810 06:52:59.380977 12214 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:59.381172 12214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:59.381587 12214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:59 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344779, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:52:59.398947 12214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "6eeffe87e2384014beaef9abb326660c"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "d170b8154a4a085f5c7b28be7ee03b77"
server_key_iv: "a4d3d94cd5f93817b2c6896169e716b8"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:59.399768 12214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "6eeffe87e2384014beaef9abb326660c"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "d170b8154a4a085f5c7b28be7ee03b77"
server_key_iv: "a4d3d94cd5f93817b2c6896169e716b8"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:59.406172 12214 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260810 06:52:59.422160 12249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:59.423528 12214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:52:59.423688 12214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "6eeffe87e2384014beaef9abb326660c"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "d170b8154a4a085f5c7b28be7ee03b77"
server_key_iv: "a4d3d94cd5f93817b2c6896169e716b8"
server_key_version: "kuduclusterkey@0"
I20260810 06:52:59.423892 12214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:52:59.448668 12214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:52:59.452471 12214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:52:59.452780 12214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:52:59.453773 12214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:52:59.455202 12214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:52:59.455286 12214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:59.455415 12214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:52:59.455469 12214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:52:59.495733 12214 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:34845
I20260810 06:52:59.495752 12460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:34845 every 8 connection(s)
I20260810 06:52:59.497146 12214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:52:59.501678 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 12214
I20260810 06:52:59.501855 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:52:59.502437 22574 external_mini_cluster.cc:1428] Setting key fb5a923f606022757651029454ca115d
Aug 10 06:52:59 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344779, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
I20260810 06:52:59.569630 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:59.503226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:37995 (local address 127.22.11.190:34173)
0810 06:52:59.523172 (+ 19946us) server_negotiation.cc:206] Beginning negotiation
0810 06:52:59.523180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:52:59.536225 (+ 13045us) server_negotiation.cc:407] Connection header received
0810 06:52:59.549783 (+ 13558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:52:59.549792 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:52:59.549925 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:52:59.550438 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:52:59.554117 (+ 3679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:59.555167 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:59.557509 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:52:59.558091 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:52:59.558190 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:59.562428 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:52:59.562435 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:52:59.562440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:52:59.562479 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:52:59.565814 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:59.566739 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:59.566745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:59.566749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:59.566917 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:52:59.567471 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:52:59.567476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:52:59.567479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:52:59.567859 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:52:59.567992 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:52:59.569012 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0810 06:52:59.569257 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19784,"thread_start_us":19960,"threads_started":1}
I20260810 06:52:59.571485 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:52:59.500767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34173 (local address 127.22.11.130:37995)
0810 06:52:59.524240 (+ 23473us) negotiation.cc:107] Waiting for socket to connect
0810 06:52:59.524311 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 06:52:59.540361 (+ 16050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:52:59.551160 (+ 10799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:52:59.551174 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:52:59.553285 (+ 2111us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:52:59.553917 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:59.553936 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:59.555370 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:59.555376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:59.557346 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:52:59.557363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:52:59.558513 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:52:59.558519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:52:59.558780 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:52:59.559488 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:52:59.559516 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:52:59.562179 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:52:59.565995 (+ 3816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:59.566005 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:59.566023 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:59.566593 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:59.567080 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:52:59.567085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:52:59.567089 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:52:59.567339 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:52:59.568099 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:52:59.568107 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:52:59.568523 (+ 416us) client_negotiation.cc:769] Sending connection context
0810 06:52:59.569945 (+ 1422us) client_negotiation.cc:240] Negotiation successful
0810 06:52:59.570387 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23089,"thread_start_us":47494,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
I20260810 06:52:59.573153 12461 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34173
I20260810 06:52:59.573458 12461 heartbeater.cc:464] Registering TS with master...
I20260810 06:52:59.574151 12461 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
I20260810 06:52:59.575764 11814 ts_manager.cc:194] Registered new tserver with Master: 6eeffe87e2384014beaef9abb326660c (127.22.11.130:34845)
I20260810 06:52:59.576822 11814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:37995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:52:59.605386 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34173
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:52:59.768441 12194 heartbeater.cc:503] Master 127.22.11.190:34173 was elected leader, sending a full tablet report...
W20260810 06:52:59.942667 12470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:52:59.942965 12470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:52:59.943111 12470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:52:59.950096 12470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:52:59.950192 12470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:52:59.950337 12470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:52:59.959928 12470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.22.11.148:56065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:52:59.961159 12470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:52:59.962415 12470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:52:59.974717 12476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:59.974783 12480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:52:59.974676 12477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:52:59.975581 12470 server_base.cc:1034] running on GCE node
I20260810 06:52:59.976181 12470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:52:59.977151 12470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:52:59.978355 12470 hybrid_clock.cc:648] HybridClock initialized: now 1786344779978301 us; error 42 us; skew 500 ppm
I20260810 06:52:59.978706 12470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:52:59 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344779, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:52:59.982862 12470 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:52:59.984553 12470 webserver.cc:460] Webserver started at http://127.22.11.131:41945/ using document root <none> and password file <none>
I20260810 06:52:59.985319 12470 fs_manager.cc:359] Metadata directory not provided
I20260810 06:52:59.985466 12470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:52:59.985765 12470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 06:52:59 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344779, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:53:00.006945 12470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "00b49127792841d3b353757558514381"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "10992f2471449494df2d90de036ab57e"
server_key_iv: "c269a1088fb9d78df69cd72e56f73cac"
server_key_version: "kuduclusterkey@0"
I20260810 06:53:00.008081 12470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "00b49127792841d3b353757558514381"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "10992f2471449494df2d90de036ab57e"
server_key_iv: "c269a1088fb9d78df69cd72e56f73cac"
server_key_version: "kuduclusterkey@0"
I20260810 06:53:00.014720 12470 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260810 06:53:00.031173 12491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:00.032984 12470 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 06:53:00.033201 12470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "00b49127792841d3b353757558514381"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "10992f2471449494df2d90de036ab57e"
server_key_iv: "c269a1088fb9d78df69cd72e56f73cac"
server_key_version: "kuduclusterkey@0"
I20260810 06:53:00.033448 12470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:00.055609 12470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:00.059008 12470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:00.059283 12470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:00.060156 12470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:00.061509 12470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:00.061584 12470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:00.061705 12470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:00.061759 12470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:00.104061 12470 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:43931
I20260810 06:53:00.104131 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:43931 every 8 connection(s)
I20260810 06:53:00.105357 12470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:53:00.114470 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 12470
I20260810 06:53:00.114657 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:53:00.115226 22574 external_mini_cluster.cc:1428] Setting key 3ab3050e5b6ebebef507baf429409f54
Aug 10 06:53:00 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344779, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:00.124069 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:00.107859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:49067 (local address 127.22.11.190:34173)
0810 06:53:00.108332 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:00.108336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:00.109275 (+ 939us) server_negotiation.cc:407] Connection header received
0810 06:53:00.111107 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:00.111113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:00.111208 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:00.111308 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:00.112714 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.113496 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.115115 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.115619 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.115729 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:00.119004 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:00.119009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:00.119013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:00.119058 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:00.121591 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.122228 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.122231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.122234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.122349 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.122749 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.122751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.122753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.122990 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:00.123101 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:00.123665 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 06:53:00.123819 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":777,"threads_started":1}
I20260810 06:53:00.124985 12677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:00.108337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34173 (local address 127.22.11.131:49067)
0810 06:53:00.109117 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:00.109159 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:00.110817 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:00.111561 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:00.111569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:00.111966 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:00.112527 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:00.112555 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.113656 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.113660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:00.114981 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:00.114993 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.115833 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.115841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:00.116063 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:00.116727 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:00.116749 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:00.118753 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:00.121715 (+ 2962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:00.121721 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:00.121733 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:00.122137 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:00.122424 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:00.122427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:00.122429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:00.122663 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:00.123079 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:00.123084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:00.123367 (+ 283us) client_negotiation.cc:769] Sending connection context
0810 06:53:00.123589 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 06:53:00.123935 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":350,"threads_started":1}
I20260810 06:53:00.126240 12672 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34173
I20260810 06:53:00.126518 12672 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:00.127206 12672 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
I20260810 06:53:00.128546 11814 ts_manager.cc:194] Registered new tserver with Master: 00b49127792841d3b353757558514381 (127.22.11.131:43931)
I20260810 06:53:00.129441 11814 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:49067
I20260810 06:53:00.130268 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:53:00.131906 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 11958
Aug 10 06:53:00 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:53:00.147929 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:00.132744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35616 (local address 127.22.11.129:43479)
0810 06:53:00.133325 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:00.133333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:00.133359 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:53:00.133419 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:00.133425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:00.133589 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:00.133716 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:00.135261 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.136229 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.137903 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.138717 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.138843 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:00.142901 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:00.142912 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:00.142924 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:00.142955 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:00.145678 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.146273 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.146278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.146281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.146412 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.146701 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.146703 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.146705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.147020 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:00.147252 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:00.147467 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 06:53:00.147664 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":251,"threads_started":1}
I20260810 06:53:00.149124 12188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:00.231189 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 11958
I20260810 06:53:00.252403 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 12214
Aug 10 06:53:00 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:53:00.268035 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:00.253430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49996 (local address 127.22.11.130:34845)
0810 06:53:00.253921 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:00.253929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:00.253953 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:53:00.254028 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:00.254034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:00.254249 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:00.254389 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:00.255702 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.256751 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.258083 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.258638 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.258750 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:00.262834 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:00.262844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:00.262857 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:00.262889 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:00.265379 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.266026 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.266033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.266035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.266157 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.266504 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.266509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.266513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.266848 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:00.267147 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:00.267449 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 06:53:00.267700 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":242,"threads_started":1}
I20260810 06:53:00.269361 12447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:00.350108 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 12214
I20260810 06:53:00.371299 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 12470
Aug 10 06:53:00 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:53:00.386547 12685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:00.372196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47748 (local address 127.22.11.131:43931)
0810 06:53:00.372808 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:00.372814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:00.372842 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:53:00.372901 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:00.372907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:00.373119 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:00.373244 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:00.374382 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.375380 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.376970 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:00.377502 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:00.377607 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:00.381556 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:00.381568 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:00.381582 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:00.381619 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:00.384273 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.384880 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.384886 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.384889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.385020 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:00.385320 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:00.385322 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:00.385324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:00.385644 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:00.385870 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:00.386066 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 06:53:00.386254 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":269,"threads_started":1}
I20260810 06:53:00.387786 12664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:00.468392 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 12470
I20260810 06:53:00.490945 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 11753
I20260810 06:53:00.492172 11876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:00.596652 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 11753
I20260810 06:53:00.631188 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34173
--webserver_interface=127.22.11.190
--webserver_port=42521
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34173
--ranger_config_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:00.956609 12688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:00.956950 12688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:00.957086 12688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:00.964115 12688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:00.964221 12688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:00.964294 12688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:00.964372 12688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:00.964438 12688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:00.974261 12688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34173
--ranger_config_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=42521
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:00.975538 12688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:00.976894 12688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:00.988178 12695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:00.988236 12697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:00.988423 12699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:00.989193 12688 server_base.cc:1034] running on GCE node
I20260810 06:53:00.989931 12688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:00.991302 12688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:00.992501 12688 hybrid_clock.cc:648] HybridClock initialized: now 1786344780992448 us; error 45 us; skew 500 ppm
I20260810 06:53:00.992856 12688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:00 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:00.995823 12688 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:53:00.997251 12688 webserver.cc:460] Webserver started at http://127.22.11.190:42521/ using document root <none> and password file <none>
I20260810 06:53:00.997897 12688 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:00.998023 12688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:53:01 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344780, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:53:01.073870 12688 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 06:53:01.076985 12715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:01.078841 12688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 06:53:01.079042 12688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "7d632c00baec4cf4b9f3b90095e1dcf8"
format_stamp: "Formatted at 2026-08-10 06:52:57 on dist-test-slave-f7th"
server_key: "3192b03351ddc67ab8ed650dfd6bb450"
server_key_iv: "3b846ca0674056f63c9a46f9a4d44539"
server_key_version: "kuduclusterkey@0"
I20260810 06:53:01.080363 12688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:01.095613 12688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:01.099192 12688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:01.099447 12688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:01.116488 12688 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34173
I20260810 06:53:01.116510 12808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34173 every 8 connection(s)
I20260810 06:53:01.117942 12688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:53:01.124744 12809 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:53:01.126070 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 12688
I20260810 06:53:01.126960 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:43479
--local_ip_for_outbound_sockets=127.22.11.129
--tserver_master_addrs=127.22.11.190:34173
--webserver_port=34719
--webserver_interface=127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:53:01.127961 12809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Bootstrap starting.
I20260810 06:53:01.145922 12809 log.cc:824] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:01.154371 12809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:53:01.154876 12809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Bootstrap complete.
I20260810 06:53:01.163952 12809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:53:01.164810 12809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d632c00baec4cf4b9f3b90095e1dcf8, State: Initialized, Role: FOLLOWER
I20260810 06:53:01.165370 12809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:53:01.165587 12809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:53:01.165699 12809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:53:01.165871 12809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:53:01.169559 12809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:53:01.169987 12809 leader_election.cc:302] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d632c00baec4cf4b9f3b90095e1dcf8; no voters:
I20260810 06:53:01.170555 12809 leader_election.cc:288] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:53:01.170691 12813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:53:01.172508 12809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:01.176880 12813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [term 2 LEADER]: Becoming Leader. State: Replica: 7d632c00baec4cf4b9f3b90095e1dcf8, State: Running, Role: LEADER
I20260810 06:53:01.177507 12813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } }
I20260810 06:53:01.180114 12817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } } }
I20260810 06:53:01.180521 12817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:01.181246 12820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d632c00baec4cf4b9f3b90095e1dcf8. Latest consensus state: current_term: 2 leader_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d632c00baec4cf4b9f3b90095e1dcf8" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34173 } } }
I20260810 06:53:01.181555 12820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:01.190961 12826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:53:01.196660 12826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:53:01.197525 12809 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 06:53:01.198176 12826 catalog_manager.cc:1259] Loaded cluster ID: 8be0e805745841d7b9a3dfbcf602e219
I20260810 06:53:01.198325 12826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:53:01.202214 12826 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:53:01.203610 12826 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Loaded TSK: 0
I20260810 06:53:01.205117 12826 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:53:01.560779 12811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:01.561163 12811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:01.561365 12811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:01.572458 12811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:01.572667 12811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:01.572904 12811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:53:01.587984 12811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:43479
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=34719
--tserver_master_addrs=127.22.11.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:01.589907 12811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:01.591737 12811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:01.609169 12879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:01.617353 12880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:01.629102 12883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:01.629735 12811 server_base.cc:1034] running on GCE node
I20260810 06:53:01.630671 12811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:01.641393 12811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:01.643689 12811 hybrid_clock.cc:648] HybridClock initialized: now 1786344781643128 us; error 568 us; skew 500 ppm
I20260810 06:53:01.644171 12811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:01 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344781, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:01.649245 12811 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:53:01.689074 12811 webserver.cc:460] Webserver started at http://127.22.11.129:34719/ using document root <none> and password file <none>
I20260810 06:53:01.690768 12811 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:01.691018 12811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:53:01 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344781, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:53:01.730511 12811 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.001s
I20260810 06:53:01.741189 12897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:01.743202 12811 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.006s sys 0.000s
I20260810 06:53:01.743487 12811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "bfb0b6231447409d9f06363dda0bc688"
format_stamp: "Formatted at 2026-08-10 06:52:58 on dist-test-slave-f7th"
server_key: "c786369cb73e4eba84613e0c7f40772e"
server_key_iv: "bbe0ca546a4bf3e6f4d1b1d7ff0d0c5e"
server_key_version: "kuduclusterkey@0"
I20260810 06:53:01.744585 12811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:01.767706 12811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:01.771617 12811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:01.771870 12811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:01.773226 12811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:01.774741 12811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:01.774838 12811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:01.774945 12811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:01.775013 12811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:01.830744 12811 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:43479
I20260810 06:53:01.830801 13088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:43479 every 8 connection(s)
I20260810 06:53:01.832650 12811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:53:01.840168 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 12811
I20260810 06:53:01.841104 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:34845
--local_ip_for_outbound_sockets=127.22.11.130
--tserver_master_addrs=127.22.11.190:34173
--webserver_port=41171
--webserver_interface=127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:53:01 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344781, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:01.872187 13093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:01.840230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34173 (local address 127.22.11.129:49587)
0810 06:53:01.845506 (+ 5276us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:01.845568 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:01.848051 (+ 2483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:01.851389 (+ 3338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:01.851411 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:01.851978 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:01.852733 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:01.852758 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:01.854937 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:01.854943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:01.857005 (+ 2062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:01.857031 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:01.858486 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:01.858493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:01.858795 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:01.859630 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:01.859664 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:01.862397 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:01.866865 (+ 4468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:01.866875 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:01.866890 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:01.867459 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:01.868021 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:01.868026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:01.868029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:01.868296 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:01.869438 (+ 1142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:01.869447 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:01.869858 (+ 411us) client_negotiation.cc:769] Sending connection context
0810 06:53:01.870170 (+ 312us) client_negotiation.cc:240] Negotiation successful
0810 06:53:01.870749 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4917,"thread_start_us":4440,"threads_started":1}
I20260810 06:53:01.874684 13095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:01.845257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:49587 (local address 127.22.11.190:34173)
0810 06:53:01.849227 (+ 3970us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:01.849245 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:01.849293 (+ 48us) server_negotiation.cc:407] Connection header received
0810 06:53:01.849718 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:01.849760 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:01.850337 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:01.850743 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:01.852999 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:01.854741 (+ 1742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:01.857190 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:01.858220 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:01.858423 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:01.862709 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:01.862720 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:01.862738 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:01.862795 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:01.866642 (+ 3847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:01.867657 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:01.867667 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:01.867675 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:01.867871 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:01.868443 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:01.868449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:01.868452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:01.869175 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:01.869364 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:01.872941 (+ 3577us) server_negotiation.cc:299] Negotiation successful
0810 06:53:01.873370 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3696,"thread_start_us":3469,"threads_started":1}
I20260810 06:53:01.878159 13090 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34173
I20260810 06:53:01.878520 13090 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:01.879477 13090 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
W20260810 06:53:01.882979 13090 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:34173 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:53:01.883219 13090 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:01.883487 13090 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
I20260810 06:53:01.884712 13090 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:01.884991 13090 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
W20260810 06:53:01.886123 13090 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:53:02.239885 13096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:02.240171 13096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:02.240330 13096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:02.249191 13096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:02.249388 13096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:02.249619 13096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:53:02.265035 13096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:34845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=41171
--tserver_master_addrs=127.22.11.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:02.266891 13096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:02.268652 13096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:02.295449 13120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:02.305290 13119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:02.305683 13124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:02.307101 13096 server_base.cc:1034] running on GCE node
I20260810 06:53:02.307884 13096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:02.309170 13096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:02.310436 13096 hybrid_clock.cc:648] HybridClock initialized: now 1786344782310351 us; error 63 us; skew 500 ppm
I20260810 06:53:02.310823 13096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:02 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:02.319231 13096 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:53:02.321329 13096 webserver.cc:460] Webserver started at http://127.22.11.130:41171/ using document root <none> and password file <none>
I20260810 06:53:02.322340 13096 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:02.322523 13096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:53:02 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:53:02.353749 13096 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
Aug 10 06:53:02 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:02.364029 13141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:02.365927 13096 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.007s sys 0.002s
I20260810 06:53:02.366156 13096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "6eeffe87e2384014beaef9abb326660c"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "d170b8154a4a085f5c7b28be7ee03b77"
server_key_iv: "a4d3d94cd5f93817b2c6896169e716b8"
server_key_version: "kuduclusterkey@0"
I20260810 06:53:02.367718 13096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:02.407248 13096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:02.412607 13096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:02.413021 13096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:02.414289 13096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:02.415906 13096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:02.416013 13096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:02.416123 13096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:02.416199 13096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:02.468137 13096 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:34845
I20260810 06:53:02.468191 13330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:34845 every 8 connection(s)
I20260810 06:53:02.469710 13096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:53:02.479079 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 13096
I20260810 06:53:02.479869 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.22.11.148:56065/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:43931
--local_ip_for_outbound_sockets=127.22.11.131
--tserver_master_addrs=127.22.11.190:34173
--webserver_port=41945
--webserver_interface=127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:33769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:53:02 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:02.498116 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:02.472712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:56251 (local address 127.22.11.190:34173)
0810 06:53:02.473213 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:02.473219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:02.477398 (+ 4179us) server_negotiation.cc:407] Connection header received
0810 06:53:02.480030 (+ 2632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:02.480037 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:02.480125 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:02.480244 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:02.482544 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:02.483725 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:02.486084 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:02.486741 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:02.486857 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:02.490772 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:02.490779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:02.490783 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:02.490825 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:02.494204 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:02.495033 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:02.495037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:02.495040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:02.495177 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:02.495664 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:02.495668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:02.495671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:02.495995 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:02.496154 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:02.497541 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0810 06:53:02.497760 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":240,"threads_started":1}
I20260810 06:53:02.498946 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:02.473300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34173 (local address 127.22.11.130:56251)
0810 06:53:02.477198 (+ 3898us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:02.477254 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:02.479690 (+ 2436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:02.481014 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:02.481025 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:02.481573 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:02.482344 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:02.482367 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:02.483936 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:02.483941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:02.485890 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:02.485916 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:02.487181 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:02.487186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:02.487373 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:02.488070 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:02.488093 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:02.490537 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:02.494349 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:02.494357 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:02.494372 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:02.494875 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:02.495290 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:02.495294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:02.495297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:02.495530 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:02.496150 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:02.496159 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:02.496576 (+ 417us) client_negotiation.cc:769] Sending connection context
0810 06:53:02.496958 (+ 382us) client_negotiation.cc:240] Negotiation successful
0810 06:53:02.497534 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3603,"thread_start_us":3382,"threads_started":1}
I20260810 06:53:02.500907 13331 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34173
I20260810 06:53:02.501243 13331 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:02.502132 13331 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
I20260810 06:53:02.504201 12751 ts_manager.cc:194] Registered new tserver with Master: 6eeffe87e2384014beaef9abb326660c (127.22.11.130:34845)
I20260810 06:53:02.506456 12751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:56251
Aug 10 06:53:02 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:53:02.886480 13090 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:02.886961 13090 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
I20260810 06:53:02.888576 12751 ts_manager.cc:194] Registered new tserver with Master: bfb0b6231447409d9f06363dda0bc688 (127.22.11.129:43479)
I20260810 06:53:02.889813 12751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:49587
W20260810 06:53:02.922427 13338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:02.922789 13338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:02.922996 13338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:02.934159 13338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:02.934293 13338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:02.934523 13338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:53:02.949607 13338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:33769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.22.11.148:56065/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:43931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=41945
--tserver_master_addrs=127.22.11.190:34173
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:02.951206 13338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:02.952924 13338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:02.969131 13351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:02.970095 13352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:02.978358 13338 server_base.cc:1034] running on GCE node
W20260810 06:53:02.978698 13355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:02.979549 13338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:02.980751 13338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:02.982035 13338 hybrid_clock.cc:648] HybridClock initialized: now 1786344782981954 us; error 67 us; skew 500 ppm
I20260810 06:53:02.982465 13338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:02 dist-test-slave-f7th krb5kdc[10902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:02.986879 13338 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:53:02.988903 13338 webserver.cc:460] Webserver started at http://127.22.11.131:41945/ using document root <none> and password file <none>
I20260810 06:53:02.989856 13338 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:02.990070 13338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 06:53:03 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:53:03.032375 13338 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.005s sys 0.005s
I20260810 06:53:03.035750 13373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
Aug 10 06:53:03 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for HTTP/127.22.11.148@KRBTEST.COM
I20260810 06:53:03.037858 13338 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 06:53:03.038034 13338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "00b49127792841d3b353757558514381"
format_stamp: "Formatted at 2026-08-10 06:52:59 on dist-test-slave-f7th"
server_key: "10992f2471449494df2d90de036ab57e"
server_key_iv: "c269a1088fb9d78df69cd72e56f73cac"
server_key_version: "kuduclusterkey@0"
I20260810 06:53:03.039517 13338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:03.072367 13338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:03.077579 13338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:03.077957 13338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:03.086191 13338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:03.092828 13338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:03.093034 13338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:03.093215 13338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:03.093371 13338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:03.212755 13338 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:43931
I20260810 06:53:03.212810 13579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:43931 every 8 connection(s)
I20260810 06:53:03.214471 13338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:53:03.223812 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 13338
Aug 10 06:53:03 dist-test-slave-f7th krb5kdc[10902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344782, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:03.286322 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.225115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55280 (local address 127.22.11.190:34173)
0810 06:53:03.258713 (+ 33598us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.258720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.258743 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:53:03.258813 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.258818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.258983 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.259099 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:03.263274 (+ 4175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.264666 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.269448 (+ 4782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.270225 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.270376 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.275650 (+ 5274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:03.275657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:03.275662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:03.275703 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:03.281021 (+ 5318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.282357 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.282364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.282367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.282514 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.283059 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.283065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.283068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.283415 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:03.283682 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.285156 (+ 1474us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.285392 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":33460,"thread_start_us":34086,"threads_started":1}
I20260810 06:53:03.294744 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.259405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:42637 (local address 127.22.11.190:34173)
0810 06:53:03.267002 (+ 7597us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.267008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.267029 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:53:03.267083 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.267088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.267187 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.269721 (+ 2534us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:03.270044 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.271300 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.279574 (+ 8274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.280297 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.280438 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.285279 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:03.285286 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:03.285291 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:03.285346 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:03.290147 (+ 4801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.291171 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.291177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.291180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.291330 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.292751 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.292756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.292759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.293128 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:03.293273 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.294159 (+ 886us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.294380 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7443,"thread_start_us":7271,"threads_started":1}
I20260810 06:53:03.296046 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.240663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34173 (local address 127.22.11.131:42637)
0810 06:53:03.242542 (+ 1879us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:03.242592 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:03.245097 (+ 2505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:03.267742 (+ 22645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:03.267759 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:03.268534 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:03.269494 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.269541 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.277251 (+ 7710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.277257 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.279368 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.279395 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.280806 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.280815 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.281109 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.281809 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:03.281839 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:03.284977 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:03.290314 (+ 5337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:03.290324 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:03.290343 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:03.291032 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:03.291468 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:03.291473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:03.291477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:03.291756 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:03.293267 (+ 1511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:03.293275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:03.293742 (+ 467us) client_negotiation.cc:769] Sending connection context
0810 06:53:03.294048 (+ 306us) client_negotiation.cc:240] Negotiation successful
0810 06:53:03.294620 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1522,"thread_start_us":1513,"threads_started":1}
I20260810 06:53:03.297384 13582 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34173
I20260810 06:53:03.297731 13582 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:03.298648 13582 heartbeater.cc:511] Master 127.22.11.190:34173 requested a full tablet report, sending...
I20260810 06:53:03.300321 12751 ts_manager.cc:194] Registered new tserver with Master: 00b49127792841d3b353757558514381 (127.22.11.131:43931)
I20260810 06:53:03.301597 12751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:42637
I20260810 06:53:03.306540 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:53:03.328786 13594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.314156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55296 (local address 127.22.11.190:34173)
0810 06:53:03.314337 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.314344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.314814 (+ 470us) server_negotiation.cc:407] Connection header received
0810 06:53:03.315025 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.315029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.315094 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.315169 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:03.317193 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.317907 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.319821 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.320229 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.320319 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.323570 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:03.323576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:03.323579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:03.323610 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:03.326278 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.327058 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.327062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.327064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.327179 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.327536 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.327539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.327541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.327790 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:03.327954 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.328297 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.328462 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:53:03.333658 12751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:53:03.341090 12751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:53:03.368453 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.356389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.22.11.129:43479)
0810 06:53:03.357560 (+ 1171us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.357569 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.358582 (+ 1013us) server_negotiation.cc:407] Connection header received
0810 06:53:03.358884 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.358892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.359143 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.359305 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:03.361216 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.363002 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.365731 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.367251 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.367393 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.367642 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.367955 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.368124 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":998,"thread_start_us":841,"threads_started":1}
I20260810 06:53:03.368499 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.355704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:43479 (local address 127.0.0.1:48730)
0810 06:53:03.358472 (+ 2768us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:03.358487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:03.358685 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:03.359638 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:03.359646 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:03.359700 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:03.360077 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.360089 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.363153 (+ 3064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.363158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.365537 (+ 2379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.365552 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.367505 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.367512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.367696 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.367744 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:53:03.367866 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 06:53:03.368067 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2653,"mutex_wait_us":30,"thread_start_us":2046,"threads_started":1}
I20260810 06:53:03.369127 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.358654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.22.11.131:43931)
0810 06:53:03.361111 (+ 2457us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.361118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.361770 (+ 652us) server_negotiation.cc:407] Connection header received
0810 06:53:03.361871 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.361878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.362083 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.362264 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:03.363214 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.364364 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.366499 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.367499 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.367633 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.367861 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.368597 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.368787 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2288,"thread_start_us":2137,"threads_started":1}
I20260810 06:53:03.369835 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.355504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:34845 (local address 127.0.0.1:37570)
0810 06:53:03.356741 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:03.356783 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:03.356965 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:03.362887 (+ 5922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:03.362892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:03.362919 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:03.363434 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.363449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.365324 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.365333 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.367444 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.367459 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.369175 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.369181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.369327 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.369351 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:53:03.369410 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 06:53:03.369528 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":946,"mutex_wait_us":24,"thread_start_us":804,"threads_started":1}
I20260810 06:53:03.370098 13625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.358155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:43931 (local address 127.0.0.1:60026)
0810 06:53:03.361526 (+ 3371us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:03.361544 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:03.361659 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:03.362597 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:03.362603 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:03.362629 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:03.363046 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.363060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.364512 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.364517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.366353 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.366365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.368018 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.368025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.368174 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.368237 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 06:53:03.368322 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:53:03.368458 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3267,"thread_start_us":3129,"threads_started":1}
I20260810 06:53:03.371253 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.356020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.22.11.130:34845)
0810 06:53:03.360639 (+ 4619us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.360648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.360674 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:53:03.360766 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.360773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.360983 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.362885 (+ 1902us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:03.363645 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.365174 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.367638 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.368927 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.370579 (+ 1652us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.370765 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.370830 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.370997 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4434,"thread_start_us":4307,"threads_started":1}
I20260810 06:53:03.373049 12981 tablet_service.cc:1467] Processing CreateTablet for tablet 07f3dc75a23e426b8de0faad3d2e96d1 (DEFAULT_TABLE table=test-table [id=0158cb407c21466094b6f1476e3366bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:03.373703 13449 tablet_service.cc:1467] Processing CreateTablet for tablet 07f3dc75a23e426b8de0faad3d2e96d1 (DEFAULT_TABLE table=test-table [id=0158cb407c21466094b6f1476e3366bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:03.374006 13215 tablet_service.cc:1467] Processing CreateTablet for tablet 07f3dc75a23e426b8de0faad3d2e96d1 (DEFAULT_TABLE table=test-table [id=0158cb407c21466094b6f1476e3366bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:03.374653 12981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f3dc75a23e426b8de0faad3d2e96d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:03.375061 13449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f3dc75a23e426b8de0faad3d2e96d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:03.375171 13215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07f3dc75a23e426b8de0faad3d2e96d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:03.387241 13627 tablet_bootstrap.cc:492] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Bootstrap starting.
I20260810 06:53:03.389809 13630 tablet_bootstrap.cc:492] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Bootstrap starting.
I20260810 06:53:03.391171 13627 tablet_bootstrap.cc:654] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:03.392457 13627 log.cc:824] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:03.393249 13628 tablet_bootstrap.cc:492] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Bootstrap starting.
I20260810 06:53:03.393914 13630 tablet_bootstrap.cc:654] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:03.395157 13630 log.cc:824] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:03.396653 13628 tablet_bootstrap.cc:654] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:03.396736 13627 tablet_bootstrap.cc:492] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: No bootstrap required, opened a new log
I20260810 06:53:03.397022 13627 ts_tablet_manager.cc:1397] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 06:53:03.397871 13628 log.cc:824] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:03.399228 13630 tablet_bootstrap.cc:492] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: No bootstrap required, opened a new log
I20260810 06:53:03.399490 13630 ts_tablet_manager.cc:1397] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260810 06:53:03.400162 13628 tablet_bootstrap.cc:492] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: No bootstrap required, opened a new log
I20260810 06:53:03.400381 13628 ts_tablet_manager.cc:1397] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 06:53:03.407079 13627 raft_consensus.cc:348] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.407519 13627 raft_consensus.cc:374] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:03.407616 13627 raft_consensus.cc:729] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eeffe87e2384014beaef9abb326660c, State: Initialized, Role: FOLLOWER
I20260810 06:53:03.407840 13628 raft_consensus.cc:348] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.408224 13628 raft_consensus.cc:374] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:03.408320 13628 raft_consensus.cc:729] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfb0b6231447409d9f06363dda0bc688, State: Initialized, Role: FOLLOWER
I20260810 06:53:03.408242 13627 consensus_queue.cc:260] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.409019 13628 consensus_queue.cc:260] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.409402 13331 heartbeater.cc:503] Master 127.22.11.190:34173 was elected leader, sending a full tablet report...
I20260810 06:53:03.410064 13627 ts_tablet_manager.cc:1428] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 06:53:03.410488 13628 ts_tablet_manager.cc:1428] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260810 06:53:03.410503 13090 heartbeater.cc:503] Master 127.22.11.190:34173 was elected leader, sending a full tablet report...
I20260810 06:53:03.411259 13630 raft_consensus.cc:348] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.411653 13630 raft_consensus.cc:374] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:03.411775 13630 raft_consensus.cc:729] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00b49127792841d3b353757558514381, State: Initialized, Role: FOLLOWER
I20260810 06:53:03.412331 13630 consensus_queue.cc:260] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.413424 13582 heartbeater.cc:503] Master 127.22.11.190:34173 was elected leader, sending a full tablet report...
I20260810 06:53:03.414013 13630 ts_tablet_manager.cc:1428] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 06:53:03.461812 13641 raft_consensus.cc:482] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:53:03.462100 13641 raft_consensus.cc:504] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.463893 13641 leader_election.cc:288] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eeffe87e2384014beaef9abb326660c (127.22.11.130:34845), bfb0b6231447409d9f06363dda0bc688 (127.22.11.129:43479)
I20260810 06:53:03.469885 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.464096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:34845 (local address 127.22.11.131:35579)
0810 06:53:03.464381 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:03.464401 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:03.464525 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:03.464903 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:03.464909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:03.464947 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:03.465139 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.465148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.466361 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.466367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.468001 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.468014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.469183 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.469189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.469359 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.469390 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:53:03.469486 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:53:03.469616 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":49}
I20260810 06:53:03.469969 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.464282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:35579 (local address 127.22.11.130:34845)
0810 06:53:03.464504 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.464510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.464531 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:53:03.464686 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.464690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.464764 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.464850 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:03.465297 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.466230 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.468173 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.469040 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.469175 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.469249 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.469541 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.469689 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:53:03.470884 13258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f3dc75a23e426b8de0faad3d2e96d1" candidate_uuid: "00b49127792841d3b353757558514381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eeffe87e2384014beaef9abb326660c" is_pre_election: true
I20260810 06:53:03.471391 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.464373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:47527 (local address 127.22.11.129:43479)
0810 06:53:03.464553 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.464559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.465207 (+ 648us) server_negotiation.cc:407] Connection header received
0810 06:53:03.465345 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.465348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.465417 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.465488 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:03.466020 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.466966 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.469414 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.470579 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.470703 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.470751 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.471084 (+ 333us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.471173 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:53:03.471531 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.464343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:43479 (local address 127.22.11.131:47527)
0810 06:53:03.465110 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:03.465124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:03.465224 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:03.465528 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:03.465533 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:03.465559 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:03.465887 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.465899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.467132 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.467137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.469183 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:03.469198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.470763 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.470769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:03.470940 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.470973 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:53:03.471073 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:53:03.471190 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":3,"thread_start_us":445,"threads_started":1}
I20260810 06:53:03.471467 13258 raft_consensus.cc:2454] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b49127792841d3b353757558514381 in term 0.
I20260810 06:53:03.472126 13374 leader_election.cc:302] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b49127792841d3b353757558514381, 6eeffe87e2384014beaef9abb326660c; no voters:
I20260810 06:53:03.472540 13641 raft_consensus.cc:2781] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:53:03.472716 13641 raft_consensus.cc:482] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:53:03.472517 13012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f3dc75a23e426b8de0faad3d2e96d1" candidate_uuid: "00b49127792841d3b353757558514381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb0b6231447409d9f06363dda0bc688" is_pre_election: true
I20260810 06:53:03.472827 13641 raft_consensus.cc:3037] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:03.473104 13012 raft_consensus.cc:2454] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00b49127792841d3b353757558514381 in term 0.
W20260810 06:53:03.473944 13333 tablet.cc:2369] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:53:03.475975 13641 raft_consensus.cc:504] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.476661 13641 leader_election.cc:288] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [CANDIDATE]: Term 1 election: Requested vote from peers 6eeffe87e2384014beaef9abb326660c (127.22.11.130:34845), bfb0b6231447409d9f06363dda0bc688 (127.22.11.129:43479)
I20260810 06:53:03.476995 13258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f3dc75a23e426b8de0faad3d2e96d1" candidate_uuid: "00b49127792841d3b353757558514381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eeffe87e2384014beaef9abb326660c"
I20260810 06:53:03.477126 13012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07f3dc75a23e426b8de0faad3d2e96d1" candidate_uuid: "00b49127792841d3b353757558514381" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfb0b6231447409d9f06363dda0bc688"
I20260810 06:53:03.477187 13258 raft_consensus.cc:3037] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:03.477326 13012 raft_consensus.cc:3037] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:03.479437 13012 raft_consensus.cc:2454] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b49127792841d3b353757558514381 in term 1.
I20260810 06:53:03.479782 13376 leader_election.cc:302] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00b49127792841d3b353757558514381, bfb0b6231447409d9f06363dda0bc688; no voters:
I20260810 06:53:03.480055 13641 raft_consensus.cc:2781] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:03.480291 13258 raft_consensus.cc:2454] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00b49127792841d3b353757558514381 in term 1.
I20260810 06:53:03.480767 13641 raft_consensus.cc:686] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 1 LEADER]: Becoming Leader. State: Replica: 00b49127792841d3b353757558514381, State: Running, Role: LEADER
I20260810 06:53:03.481204 13641 consensus_queue.cc:237] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } }
I20260810 06:53:03.485419 12751 catalog_manager.cc:5495] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00b49127792841d3b353757558514381 (127.22.11.131). New cstate: current_term: 1 leader_uuid: "00b49127792841d3b353757558514381" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00b49127792841d3b353757558514381" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 43931 } health_report { overall_health: HEALTHY } } }
W20260810 06:53:03.517810 13585 tablet.cc:2369] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:53:03.528473 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.523475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60042 (local address 127.22.11.131:43931)
0810 06:53:03.523656 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.523661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.523906 (+ 245us) server_negotiation.cc:407] Connection header received
0810 06:53:03.524008 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.524012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.524091 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.524220 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:53:03.524513 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.525326 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.526609 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.527037 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.527134 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.527521 (+ 387us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:53:03.528063 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:53:03.528177 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.528215 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.528277 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:53:03.536273 13012 raft_consensus.cc:1261] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 1 FOLLOWER]: Refusing update from remote peer 00b49127792841d3b353757558514381: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:03.537151 13644 consensus_queue.cc:1035] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfb0b6231447409d9f06363dda0bc688" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 43479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:03.537302 13258 raft_consensus.cc:1261] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 1 FOLLOWER]: Refusing update from remote peer 00b49127792841d3b353757558514381: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:03.538044 13644 consensus_queue.cc:1035] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eeffe87e2384014beaef9abb326660c" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:03.548915 13650 mvcc.cc:205] Tried to move back new op lower bound from 7316868233355042816 to 7316868233142267904. Current Snapshot: MvccSnapshot[applied={T|T < 7316868233355042816}]
I20260810 06:53:03.561484 12747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:03.561820 12747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:03.566931 12747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d632c00baec4cf4b9f3b90095e1dcf8: Sending DeleteTablet for 3 replicas of tablet 07f3dc75a23e426b8de0faad3d2e96d1
I20260810 06:53:03.568104 13215 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f3dc75a23e426b8de0faad3d2e96d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:37570
I20260810 06:53:03.568892 13662 tablet_replica.cc:330] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: stopping tablet replica
I20260810 06:53:03.569219 13662 raft_consensus.cc:2229] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:03.569597 13662 raft_consensus.cc:2258] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:03.571234 13662 ts_tablet_manager.cc:1905] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:03.573439 12981 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f3dc75a23e426b8de0faad3d2e96d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:48730
I20260810 06:53:03.573539 13449 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f3dc75a23e426b8de0faad3d2e96d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:60026
I20260810 06:53:03.573989 12979 tablet_service.cc:1514] Processing DeleteTablet for tablet 07f3dc75a23e426b8de0faad3d2e96d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:48730
I20260810 06:53:03.577572 13662 ts_tablet_manager.cc:1918] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:03.577736 13662 log.cc:1196] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/07f3dc75a23e426b8de0faad3d2e96d1
I20260810 06:53:03.578197 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 12811
I20260810 06:53:03.578270 13662 ts_tablet_manager.cc:1939] T 07f3dc75a23e426b8de0faad3d2e96d1 P 6eeffe87e2384014beaef9abb326660c: Deleting consensus metadata
I20260810 06:53:03.580335 12718 catalog_manager.cc:4841] TS 6eeffe87e2384014beaef9abb326660c (127.22.11.130:34845): tablet 07f3dc75a23e426b8de0faad3d2e96d1 (table test-table [id=0158cb407c21466094b6f1476e3366bf]) successfully deleted
I20260810 06:53:03.581087 13671 tablet_replica.cc:330] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: stopping tablet replica
I20260810 06:53:03.581415 13671 raft_consensus.cc:2229] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:03.581799 13671 raft_consensus.cc:2258] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:03.583433 13671 ts_tablet_manager.cc:1905] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:03.590838 13671 ts_tablet_manager.cc:1918] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:03.591029 13671 log.cc:1196] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/07f3dc75a23e426b8de0faad3d2e96d1
I20260810 06:53:03.591622 13671 ts_tablet_manager.cc:1939] T 07f3dc75a23e426b8de0faad3d2e96d1 P bfb0b6231447409d9f06363dda0bc688: Deleting consensus metadata
I20260810 06:53:03.592965 13672 tablet_replica.cc:330] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: stopping tablet replica
I20260810 06:53:03.593458 12720 catalog_manager.cc:4841] TS bfb0b6231447409d9f06363dda0bc688 (127.22.11.129:43479): tablet 07f3dc75a23e426b8de0faad3d2e96d1 (table test-table [id=0158cb407c21466094b6f1476e3366bf]) successfully deleted
I20260810 06:53:03.593533 13672 raft_consensus.cc:2229] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:53:03.594329 13672 raft_consensus.cc:2258] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260810 06:53:03.594475 12720 catalog_manager.cc:4808] TS bfb0b6231447409d9f06363dda0bc688 (127.22.11.129:43479): delete failed for tablet 07f3dc75a23e426b8de0faad3d2e96d1 because the tablet was not found. No further retry: Not found: Tablet not found: 07f3dc75a23e426b8de0faad3d2e96d1
I20260810 06:53:03.596064 13672 ts_tablet_manager.cc:1905] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:03.599525 13624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.579116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.22.11.129:43479)
0810 06:53:03.579672 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.579678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.579695 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:53:03.579747 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.579753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.579834 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.579942 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:03.582299 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.583473 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.586879 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.587524 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.587649 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.591254 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:03.591268 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:03.591287 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:03.591325 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:03.595354 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.596273 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.596282 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.596285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.596444 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.596949 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.596954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.596957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.597438 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:03.597741 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.598910 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.599130 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438}
I20260810 06:53:03.600286 13081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:03.602922 13672 ts_tablet_manager.cc:1918] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:03.603295 13672 log.cc:1196] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/07f3dc75a23e426b8de0faad3d2e96d1
I20260810 06:53:03.604369 13672 ts_tablet_manager.cc:1939] T 07f3dc75a23e426b8de0faad3d2e96d1 P 00b49127792841d3b353757558514381: Deleting consensus metadata
I20260810 06:53:03.606143 12720 catalog_manager.cc:4841] TS 00b49127792841d3b353757558514381 (127.22.11.131:43931): tablet 07f3dc75a23e426b8de0faad3d2e96d1 (table test-table [id=0158cb407c21466094b6f1476e3366bf]) successfully deleted
I20260810 06:53:03.690894 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 12811
I20260810 06:53:03.724263 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 13096
I20260810 06:53:03.739136 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.725087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.22.11.130:34845)
0810 06:53:03.725266 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.725271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.725289 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:53:03.725481 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.725484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.725548 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.725628 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:03.727297 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.728005 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.729877 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.730259 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.730352 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.733549 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:03.733563 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:03.733580 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:03.733614 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:03.736320 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.737150 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.737157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.737159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.737302 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.737708 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.737711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.737713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.738056 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:03.738269 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.738644 (+ 375us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.738833 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:53:03.739804 13325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:03.827468 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 13096
I20260810 06:53:03.847318 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 13338
I20260810 06:53:03.861229 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:03.848155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.22.11.131:43931)
0810 06:53:03.848320 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:03.848325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:03.848346 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:53:03.848556 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:03.848560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:03.848665 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:03.848757 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:03.850061 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.850868 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.852138 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:03.852572 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:03.852804 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:03.855238 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:03.855249 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:03.855264 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:03.855303 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:03.858509 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.859141 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.859148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.859150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.859292 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:03.859653 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:03.859656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:03.859658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:03.860044 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:03.860375 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:03.860664 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 06:53:03.860880 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:53:03.861933 13570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:03.980195 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 13338
I20260810 06:53:04.006783 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 12688
I20260810 06:53:04.007987 12802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:04.119716 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 12688
I20260810 06:53:04.149019 22574 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 06:53:04.493364 22574 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 06:53:04.493563 22574 mini_ranger.cc:66] Stopping Ranger...
I20260810 06:53:04.837309 22574 mini_ranger.cc:68] Stopped Ranger
2026-08-10 06:53:04.837 UTC [10960] LOG: received smart shutdown request
2026-08-10 06:53:04.840 UTC [10960] LOG: background worker "logical replication launcher" (PID 10977) exited with exit code 1
2026-08-10 06:53:04.840 UTC [10967] LOG: shutting down
2026-08-10 06:53:04.966 UTC [10960] LOG: database system is shut down
2026-08-10T06:53:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50757 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:53:05 dist-test-slave-f7th krb5kdc[13694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:53:05 dist-test-slave-f7th krb5kdc[13694](info): set up 2 sockets
Aug 10 06:53:05 dist-test-slave-f7th krb5kdc[13694](info): commencing operation
krb5kdc: starting...
W20260810 06:53:06.312067 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:53:06 dist-test-slave-f7th krb5kdc[13694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:53:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:53:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:53:06.563723 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34567
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34567,127.22.11.189:46271,127.22.11.188:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:06.892413 13713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:06.892706 13713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:06.892844 13713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:06.899861 13713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:06.899960 13713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:06.900051 13713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:06.900125 13713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:06.900182 13713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:06.910106 13713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34567,127.22.11.189:46271,127.22.11.188:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:06.911391 13713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:06.912757 13713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:06.923911 13722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:06.923996 13721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:06.923960 13726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:06.924681 13713 server_base.cc:1034] running on GCE node
I20260810 06:53:06.925397 13713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:06.926723 13713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:06.927917 13713 hybrid_clock.cc:648] HybridClock initialized: now 1786344786927874 us; error 51 us; skew 500 ppm
I20260810 06:53:06.928241 13713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:06 dist-test-slave-f7th krb5kdc[13694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:06.931244 13713 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:53:06.932542 13713 webserver.cc:460] Webserver started at http://127.22.11.190:38455/ using document root <none> and password file <none>
I20260810 06:53:06.933188 13713 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:06.933290 13713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:06.933545 13713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:06.936383 13713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "0682a48b7d444e98949293c1c1f749f9"
format_stamp: "Formatted at 2026-08-10 06:53:06 on dist-test-slave-f7th"
server_key: "3f972fed0d383e29e34cb9b294c7e683"
server_key_iv: "5b42bb91b31ed158660441d031f0cd90"
server_key_version: "encryptionkey@0"
I20260810 06:53:06.937172 13713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "0682a48b7d444e98949293c1c1f749f9"
format_stamp: "Formatted at 2026-08-10 06:53:06 on dist-test-slave-f7th"
server_key: "3f972fed0d383e29e34cb9b294c7e683"
server_key_iv: "5b42bb91b31ed158660441d031f0cd90"
server_key_version: "encryptionkey@0"
I20260810 06:53:06.941943 13713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260810 06:53:06.945179 13736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:06.946408 13713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:53:06.946559 13713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "0682a48b7d444e98949293c1c1f749f9"
format_stamp: "Formatted at 2026-08-10 06:53:06 on dist-test-slave-f7th"
server_key: "3f972fed0d383e29e34cb9b294c7e683"
server_key_iv: "5b42bb91b31ed158660441d031f0cd90"
server_key_version: "encryptionkey@0"
I20260810 06:53:06.946713 13713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:06.964285 13713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:06.967978 13713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:06.968199 13713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:06.985639 13713 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34567
I20260810 06:53:06.985698 13822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34567 every 8 connection(s)
I20260810 06:53:06.987119 13713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:53:06.990816 13823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:06.996933 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 13713
I20260810 06:53:06.997150 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:53:06.997063 13823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:53:06.997829 22574 external_mini_cluster.cc:1428] Setting key 15bd05c727121403c9669398beedcca9
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:07.018740 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:06.999968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.22.11.190:34567)
0810 06:53:07.001641 (+ 1673us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.001649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.001699 (+ 50us) server_negotiation.cc:407] Connection header received
0810 06:53:07.003189 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.003195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.003413 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.003591 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.004880 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.006066 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.007417 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.008031 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.008203 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.011827 (+ 3624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.011834 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.011845 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.011878 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.014645 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.015491 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.015497 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.015499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.015657 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.016069 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.016072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.016074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.016550 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.016749 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.017346 (+ 597us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.017650 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":1445,"spinlock_wait_cycles":15616,"thread_start_us":1240,"threads_started":1}
I20260810 06:53:07.018766 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:06.999967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34567 (local address 127.0.0.1:36818)
0810 06:53:07.000843 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.000887 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:07.002822 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:07.003643 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:07.003650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:07.004325 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:07.004702 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.004722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.006216 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.006220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.007287 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.007299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.008251 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.008259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.008485 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.009100 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:07.009121 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:07.011594 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:07.014781 (+ 3187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.014793 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.014804 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.015360 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.015761 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.015765 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.015768 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.015962 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.016720 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:07.016728 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:07.016969 (+ 241us) client_negotiation.cc:769] Sending connection context
0810 06:53:07.017234 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 06:53:07.017794 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":145,"thread_start_us":364,"threads_started":1}
I20260810 06:53:07.021546 13823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:53:07.022655 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.022205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38220)
0810 06:53:07.022394 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.022524 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 06:53:07.022961 13740 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.11.189@KRBTEST.COM; defaulting to no policy
I20260810 06:53:07.024441 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.024113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38224)
0810 06:53:07.024251 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.024320 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:53:07.024681 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:46271: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab.
Entry for principal kudu/127.22.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab.
I20260810 06:53:07.054062 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:46271
--webserver_interface=127.22.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab
--principal=kudu/127.22.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34567,127.22.11.189:46271,127.22.11.188:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:53:07.074047 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } attempt: 1
I20260810 06:53:07.075249 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.074771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38230)
0810 06:53:07.074995 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.075108 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:53:07.076263 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.075908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38238)
0810 06:53:07.076061 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.076133 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:53:07.076457 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:46271: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
I20260810 06:53:07.144704 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } attempt: 2
I20260810 06:53:07.145848 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.145350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38246)
0810 06:53:07.145604 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.145709 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 06:53:07.146940 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.146569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38258)
0810 06:53:07.146708 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.146786 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:53:07.147127 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:46271: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
I20260810 06:53:07.238372 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } attempt: 3
I20260810 06:53:07.239634 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.239160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38266)
0810 06:53:07.239379 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.239491 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 06:53:07.240710 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.240295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38278)
0810 06:53:07.240448 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.240520 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:53:07.240911 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:46271: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
W20260810 06:53:07.379643 13834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:07.379922 13834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:07.380059 13834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 06:53:07.384133 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } attempt: 4
I20260810 06:53:07.385252 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.384735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38280)
0810 06:53:07.385025 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.385114 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
I20260810 06:53:07.386242 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.385882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38296)
0810 06:53:07.386044 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.386124 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:53:07.386421 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:46271: Network error: Client connection negotiation failed: client connection to 127.22.11.189:46271: connect: Connection refused (error 111)
W20260810 06:53:07.387363 13834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:07.387467 13834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:07.387538 13834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:07.387617 13834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:07.387676 13834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:07.397190 13834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34567,127.22.11.189:46271,127.22.11.188:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab
--principal=kudu/127.22.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:46271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:07.398444 13834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:07.399647 13834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:07.410718 13849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:07.410789 13845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:07.410764 13847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:07.411554 13834 server_base.cc:1034] running on GCE node
I20260810 06:53:07.412271 13834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:07.413589 13834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:07.414784 13834 hybrid_clock.cc:648] HybridClock initialized: now 1786344787414736 us; error 36 us; skew 500 ppm
I20260810 06:53:07.415122 13834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:07.420807 13834 init.cc:377] Logged in from keytab as kudu/127.22.11.189@KRBTEST.COM (short username kudu)
I20260810 06:53:07.422166 13834 webserver.cc:460] Webserver started at http://127.22.11.189:38075/ using document root <none> and password file <none>
I20260810 06:53:07.422806 13834 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:07.422930 13834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:07.423206 13834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:07.426107 13834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data/instance:
uuid: "da29e98e06e94fb690e59de497a5039a"
format_stamp: "Formatted at 2026-08-10 06:53:07 on dist-test-slave-f7th"
server_key: "0df9accccf99e7a5c13968db15a62415"
server_key_iv: "f916a043642f8095fde0b44c8e960615"
server_key_version: "encryptionkey@0"
I20260810 06:53:07.426896 13834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/wal/instance:
uuid: "da29e98e06e94fb690e59de497a5039a"
format_stamp: "Formatted at 2026-08-10 06:53:07 on dist-test-slave-f7th"
server_key: "0df9accccf99e7a5c13968db15a62415"
server_key_iv: "f916a043642f8095fde0b44c8e960615"
server_key_version: "encryptionkey@0"
I20260810 06:53:07.431612 13834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260810 06:53:07.434779 13860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:07.435973 13834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:53:07.436131 13834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/wal
uuid: "da29e98e06e94fb690e59de497a5039a"
format_stamp: "Formatted at 2026-08-10 06:53:07 on dist-test-slave-f7th"
server_key: "0df9accccf99e7a5c13968db15a62415"
server_key_iv: "f916a043642f8095fde0b44c8e960615"
server_key_version: "encryptionkey@0"
I20260810 06:53:07.436295 13834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:07.448871 13834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:07.452458 13834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:07.452790 13834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:07.469518 13834 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.189:46271
I20260810 06:53:07.469558 13958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.189:46271 every 8 connection(s)
I20260810 06:53:07.470767 13834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
I20260810 06:53:07.474166 13959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:07.477494 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 13834
I20260810 06:53:07.477670 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-1/wal/instance
I20260810 06:53:07.478094 22574 external_mini_cluster.cc:1428] Setting key 27d386e6e5b3cd8feb1342f13f8c0e3f
I20260810 06:53:07.480224 13959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:07.498772 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.482594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36834 (local address 127.22.11.190:34567)
0810 06:53:07.482758 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.482763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.483959 (+ 1196us) server_negotiation.cc:407] Connection header received
0810 06:53:07.485794 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.485800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.485879 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.485967 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.487385 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.488067 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.489618 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.490028 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.490140 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.493434 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.493439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.493442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.493474 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.495889 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.496568 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.496572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.496575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.496725 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.497132 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.497137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.497141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.497410 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.497550 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.498211 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.498440 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:53:07.499679 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.483112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34567 (local address 127.0.0.1:36834)
0810 06:53:07.483815 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.483850 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:07.485536 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:07.486196 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:07.486204 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:07.486675 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:07.487206 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.487223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.488220 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.488224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.489469 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.489482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.490234 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.490242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.490441 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.491048 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:07.491073 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:07.493226 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:07.496028 (+ 2802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.496037 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.496051 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.496441 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.496827 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.496832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.496836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.497016 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.497533 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:07.497540 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:07.497838 (+ 298us) client_negotiation.cc:769] Sending connection context
0810 06:53:07.498067 (+ 229us) client_negotiation.cc:240] Negotiation successful
0810 06:53:07.498649 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":308,"threads_started":1}
I20260810 06:53:07.501305 13959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.11.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.188@KRBTEST.COM" created.
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:53:07.515231 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.501978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38300)
0810 06:53:07.502195 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.502214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:07.502320 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:07.503218 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:07.503225 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:07.503544 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:07.503678 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.503686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.505244 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.505249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.506430 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.506445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.507231 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.507236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.507405 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.507904 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:07.507924 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:07.510085 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:07.513018 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.513023 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.513027 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.513373 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.513702 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.513707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.513711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.513853 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.514552 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:07.514556 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:07.514666 (+ 110us) client_negotiation.cc:769] Sending connection context
0810 06:53:07.514794 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 06:53:07.514982 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 06:53:07.515616 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.502247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.22.11.189:46271)
0810 06:53:07.502814 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.502820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.502845 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:53:07.502887 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.502892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.503073 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.503177 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.503809 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.505114 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.506564 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.507098 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.507190 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.510272 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.510282 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.510294 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.510321 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.512908 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.513492 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.513499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.513501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.513605 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.513950 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.513954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.513958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.514387 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.514594 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.514966 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.515225 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":310,"threads_started":1}
I20260810 06:53:07.517976 13959 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:53:07.519083 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.518671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49892)
0810 06:53:07.518808 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.518916 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:53:07.519425 13865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
I20260810 06:53:07.520884 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.520557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49898)
0810 06:53:07.520725 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.520796 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:53:07.521090 13959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab.
Entry for principal kudu/127.22.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab.
I20260810 06:53:07.535916 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:44571
--webserver_interface=127.22.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab
--principal=kudu/127.22.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:36107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34567,127.22.11.189:46271,127.22.11.188:44571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:53:07.570356 13959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 1
I20260810 06:53:07.571444 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.571024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49910)
0810 06:53:07.571206 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.571299 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:53:07.572422 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.572124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49924)
0810 06:53:07.572246 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.572321 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:53:07.572636 13959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
I20260810 06:53:07.640873 13959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 2
I20260810 06:53:07.641987 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.641544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49928)
0810 06:53:07.641728 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.641823 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:53:07.642968 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.642626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49934)
0810 06:53:07.642754 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.642844 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:53:07.643183 13959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
I20260810 06:53:07.685678 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } attempt: 5
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:53:07.698544 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.686432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38312)
0810 06:53:07.686649 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.686672 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:07.686823 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:07.687233 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:07.687238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:07.687630 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:07.687816 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.687825 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.688913 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.688917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.689782 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.689794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.690537 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.690548 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.690731 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.691402 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:07.691419 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:07.693449 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:07.696208 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.696214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.696218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.696585 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.696950 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.696955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.696958 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.697093 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.697603 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:07.697608 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:07.697760 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 06:53:07.697894 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:53:07.698179 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:53:07.698536 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.686504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38312 (local address 127.22.11.189:46271)
0810 06:53:07.686680 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.686685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.686757 (+ 72us) server_negotiation.cc:407] Connection header received
0810 06:53:07.686999 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.687004 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.687087 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.687175 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.687997 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.688769 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.689940 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.690392 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.690507 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.693636 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.693641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.693644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.693677 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.696083 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.696707 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.696713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.696717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.696846 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.697194 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.697199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.697203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.697488 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.697623 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.698008 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.698219 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:53:07.699678 13823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:53:07.700732 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.700313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49938)
0810 06:53:07.700484 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.700565 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:53:07.701620 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.701305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49940)
0810 06:53:07.701442 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.701525 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:53:07.701807 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
I20260810 06:53:07.734400 13959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 3
I20260810 06:53:07.735602 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.735091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49950)
0810 06:53:07.735326 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.735435 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 06:53:07.736639 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.736194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49964)
0810 06:53:07.736431 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.736508 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260810 06:53:07.736881 13959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
I20260810 06:53:07.753015 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 1
I20260810 06:53:07.753950 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.753529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49978)
0810 06:53:07.753702 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.753796 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 06:53:07.754909 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.754601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:49994)
0810 06:53:07.754739 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.754804 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:53:07.755095 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
I20260810 06:53:07.823283 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 2
I20260810 06:53:07.824339 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.823876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50006)
0810 06:53:07.824078 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.824179 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 06:53:07.825348 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.825020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50020)
0810 06:53:07.825180 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.825245 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:53:07.825553 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
W20260810 06:53:07.857496 13972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:07.857779 13972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:07.857913 13972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:07.864908 13972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:07.865013 13972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:07.865096 13972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:07.865170 13972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:07.865227 13972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:07.875125 13972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:36107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34567,127.22.11.189:46271,127.22.11.188:44571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab
--principal=kudu/127.22.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:44571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:07.876381 13972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:07.877696 13972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 06:53:07.880074 13959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 4
I20260810 06:53:07.881228 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.880774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50036)
0810 06:53:07.880993 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.881079 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:53:07.882149 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.881848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50038)
0810 06:53:07.881987 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.882051 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:53:07.882426 13959 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
W20260810 06:53:07.888762 13991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:07.888808 13992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:07.888831 13997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:07.889603 13972 server_base.cc:1034] running on GCE node
I20260810 06:53:07.890269 13972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:07.891490 13972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:07.892704 13972 hybrid_clock.cc:648] HybridClock initialized: now 1786344787892649 us; error 39 us; skew 500 ppm
I20260810 06:53:07.893028 13972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:07.895972 13972 init.cc:377] Logged in from keytab as kudu/127.22.11.188@KRBTEST.COM (short username kudu)
I20260810 06:53:07.897442 13972 webserver.cc:460] Webserver started at http://127.22.11.188:42655/ using document root <none> and password file <none>
I20260810 06:53:07.898074 13972 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:07.898188 13972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:07.898463 13972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:07.901345 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data/instance:
uuid: "c2d001a875e9404c8fb5969a93c6f30c"
format_stamp: "Formatted at 2026-08-10 06:53:07 on dist-test-slave-f7th"
server_key: "2f5a3d2d113da5b0cb53136219eead4d"
server_key_iv: "662eccf91e1649e96c19c70ee2f15aa6"
server_key_version: "encryptionkey@0"
I20260810 06:53:07.902109 13972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/wal/instance:
uuid: "c2d001a875e9404c8fb5969a93c6f30c"
format_stamp: "Formatted at 2026-08-10 06:53:07 on dist-test-slave-f7th"
server_key: "2f5a3d2d113da5b0cb53136219eead4d"
server_key_iv: "662eccf91e1649e96c19c70ee2f15aa6"
server_key_version: "encryptionkey@0"
I20260810 06:53:07.906944 13972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:53:07.910288 14006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:07.911518 13972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:53:07.911665 13972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/wal
uuid: "c2d001a875e9404c8fb5969a93c6f30c"
format_stamp: "Formatted at 2026-08-10 06:53:07 on dist-test-slave-f7th"
server_key: "2f5a3d2d113da5b0cb53136219eead4d"
server_key_iv: "662eccf91e1649e96c19c70ee2f15aa6"
server_key_version: "encryptionkey@0"
I20260810 06:53:07.911832 13972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:07.926592 13972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:07.930619 13972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:07.930866 13972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:07.931754 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 3
I20260810 06:53:07.932756 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.932279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50046)
0810 06:53:07.932483 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.932568 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 06:53:07.933764 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.933367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50052)
0810 06:53:07.933570 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.933653 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260810 06:53:07.933970 13823 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44571: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44571: connect: Connection refused (error 111)
I20260810 06:53:07.948030 13972 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.188:44571
I20260810 06:53:07.948082 14089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.188:44571 every 8 connection(s)
I20260810 06:53:07.949342 13972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
I20260810 06:53:07.953014 14093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:07.957939 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 13972
I20260810 06:53:07.958139 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786344580063516-22574-0/minicluster-data/master-2/wal/instance
I20260810 06:53:07.958740 22574 external_mini_cluster.cc:1428] Setting key 057017073b178f9ae179394833c48767
I20260810 06:53:07.959841 14093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
I20260810 06:53:07.977749 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.962433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36838 (local address 127.22.11.190:34567)
0810 06:53:07.962599 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.962604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.963148 (+ 544us) server_negotiation.cc:407] Connection header received
0810 06:53:07.963698 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.963704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.963783 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.963882 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.965219 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.965986 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.967676 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.968307 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.968396 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.971920 (+ 3524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.971925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.971928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.971959 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.974810 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.975605 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.975609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.975611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.975733 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.976151 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.976153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.976155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.976424 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.976572 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.977263 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.977480 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:53:07.978794 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.962433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34567 (local address 127.0.0.1:36838)
0810 06:53:07.963020 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.963051 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:07.963466 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:07.963980 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:07.963986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:07.964636 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:07.965060 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.965078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.966146 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.966151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.967543 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.967555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.968513 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.968524 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.968898 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.969615 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:07.969636 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:07.971738 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:07.974931 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.974943 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.974955 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.975499 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.975832 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.975837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.975844 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.976063 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.976553 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:07.976559 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:07.976866 (+ 307us) client_negotiation.cc:769] Sending connection context
0810 06:53:07.977162 (+ 296us) client_negotiation.cc:240] Negotiation successful
0810 06:53:07.977520 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":43,"thread_start_us":255,"threads_started":1}
I20260810 06:53:07.979144 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.960515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50054 (local address 127.22.11.188:44571)
0810 06:53:07.961263 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.961273 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.961316 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:53:07.963463 (+ 2147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.963484 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.963694 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.963842 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.965406 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.966808 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.968247 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.969079 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.969206 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.973366 (+ 4160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.973374 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.973384 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.973411 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.976209 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.976948 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.976956 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.976959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.977112 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.977519 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.977524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.977526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.977944 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.978233 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.978484 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.978706 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":543,"thread_start_us":316,"threads_started":1}
I20260810 06:53:07.981122 14093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
Aug 10 06:53:07 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:07.993911 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.981759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:46271 (local address 127.0.0.1:38328)
0810 06:53:07.981955 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.981968 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:07.982061 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:07.982335 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:07.982339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:07.982675 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:07.982805 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.982813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.983736 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.983741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.984569 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.984579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.985312 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.985318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.985512 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.985961 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:07.985982 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:07.988179 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:07.991318 (+ 3139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.991323 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.991327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.991808 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.992221 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:07.992226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:07.992229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:07.992406 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:07.992983 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:07.992988 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:07.993151 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 06:53:07.993302 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 06:53:07.993565 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 06:53:07.994421 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.981775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38328 (local address 127.22.11.189:46271)
0810 06:53:07.981954 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.981960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.982050 (+ 90us) server_negotiation.cc:407] Connection header received
0810 06:53:07.982159 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.982165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.982225 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.982303 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.982950 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.983610 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.984720 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.985183 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.985291 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.988351 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.988355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.988358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.988390 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.991172 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.991930 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.991936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.991939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.992098 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.992543 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.992548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.992552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.992876 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.992988 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.993425 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.993653 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:53:07.995424 14093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:53:07.998183 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.981774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36840 (local address 127.22.11.190:34567)
0810 06:53:07.982077 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.982082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.982097 (+ 15us) server_negotiation.cc:407] Connection header received
0810 06:53:07.982214 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.982219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.982273 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.982345 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.983424 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.984130 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.985739 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.986354 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.986478 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:07.991834 (+ 5356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:07.991838 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:07.991841 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:07.991872 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:07.995647 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.996372 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.996377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.996380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.996504 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:07.996876 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:07.996881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:07.996884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:07.997144 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:07.997390 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:07.997714 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 06:53:07.997918 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
Aug 10 06:53:08 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
Aug 10 06:53:08 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:53:08.009151 14103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.996195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50064)
0810 06:53:07.996413 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:07.996426 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:07.996520 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:07.996791 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:07.996796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:07.997147 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:07.997275 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.997282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.998650 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.998655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:07.999622 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:07.999638 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.000493 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.000498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:08.000732 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.001189 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:08.001205 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:08.003603 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:08.006763 (+ 3160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:08.006769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:08.006773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:08.007252 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:08.007569 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:08.007574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:08.007578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:08.007729 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:08.008200 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:08.008205 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:08.008335 (+ 130us) client_negotiation.cc:769] Sending connection context
0810 06:53:08.008455 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:53:08.008744 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":21}
I20260810 06:53:08.009172 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.996141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.22.11.188:44571)
0810 06:53:07.996309 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.996314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.996484 (+ 170us) server_negotiation.cc:407] Connection header received
0810 06:53:07.996632 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.996637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:07.996689 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:07.996761 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:07.997663 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:07.998527 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:07.999770 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.000360 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.000484 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.003762 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:08.003767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:08.003770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:08.003798 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:08.006624 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.007359 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.007363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.007365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.007476 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.007829 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.007832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.007834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.008098 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:08.008221 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:08.008560 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 06:53:08.008894 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":138,"server-negotiator.queue_time_us":56}
I20260810 06:53:08.014034 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:07.999675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38332 (local address 127.22.11.189:46271)
0810 06:53:07.999820 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:07.999824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:07.999838 (+ 14us) server_negotiation.cc:407] Connection header received
0810 06:53:07.999957 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:07.999960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:08.000009 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:08.000081 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:08.001430 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.002453 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.004128 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.004709 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.004833 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.008894 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:08.008899 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:08.008902 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:08.008931 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:08.011616 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.012331 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.012334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.012336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.012453 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.012867 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.012870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.012872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.013107 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:08.013310 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:08.013608 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 06:53:08.013767 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:53:08.014777 14093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: Bootstrap starting.
I20260810 06:53:08.018016 14093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:08.018630 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:53:08.019132 14093 log.cc:824] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:08.021663 14093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: No bootstrap required, opened a new log
I20260810 06:53:08.032526 14093 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.032974 14093 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:08.033080 14093 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2d001a875e9404c8fb5969a93c6f30c, State: Initialized, Role: FOLLOWER
I20260810 06:53:08.033776 14093 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.036445 14120 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.038650 14120 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:53:08.038528 14093 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:08.053406 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:08.023651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.22.11.190:34567)
0810 06:53:08.023796 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:08.023801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:08.034829 (+ 11028us) server_negotiation.cc:407] Connection header received
0810 06:53:08.035022 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:08.035028 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:08.035127 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:08.035220 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:08.037685 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.038734 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.040893 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.041522 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.041614 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.046143 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:08.046149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:08.046152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:08.046183 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:08.048955 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.050059 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.050065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.050068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.050229 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.050734 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.050739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.050742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.051136 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:08.051372 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:08.052827 (+ 1455us) server_negotiation.cc:299] Negotiation successful
0810 06:53:08.053056 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260810 06:53:08.054610 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:08.023802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38346 (local address 127.22.11.189:46271)
0810 06:53:08.024071 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:08.024077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:08.035488 (+ 11411us) server_negotiation.cc:407] Connection header received
0810 06:53:08.035740 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:08.035745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:08.035815 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:08.035920 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:08.038324 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.039345 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.041813 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.042467 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.042572 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.046487 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:08.046504 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:08.046510 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:08.046551 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:08.050724 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.051601 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.051606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.051609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.051763 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.052278 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.052283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.052286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.052728 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:08.052988 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:08.053970 (+ 982us) server_negotiation.cc:299] Negotiation successful
0810 06:53:08.054220 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168}
I20260810 06:53:08.055909 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:08.023385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.22.11.188:44571)
0810 06:53:08.023947 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:08.023954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:08.035788 (+ 11834us) server_negotiation.cc:407] Connection header received
0810 06:53:08.035946 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:08.035950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:08.036032 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:08.036133 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:08.038216 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.039282 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.041450 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.042161 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.042255 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.047361 (+ 5106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:08.047369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:08.047374 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:08.047414 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:08.051431 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.052672 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.052678 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.052681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.052877 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.053338 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.053342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.053345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.053751 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:08.053965 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:08.055340 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0810 06:53:08.055557 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 06:53:08.063758 14138 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:53:08.063895 14138 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:53:08.060847 13972 master_runner.cc:186] Error getting master registration for 127.22.11.190:34567: OK, instance_id { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" instance_seqno: 1786344786980472 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:53:08.072300 13972 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:53:08.111197 13823 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 4
Aug 10 06:53:08 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344786, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
I20260810 06:53:08.125634 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:08.111810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50082 (local address 127.22.11.188:44571)
0810 06:53:08.111960 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:08.111964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:08.112013 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:53:08.112187 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:08.112191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:08.112270 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:08.112346 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:08.113274 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.114061 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.115839 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.116586 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.116730 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.120349 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:08.120356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:08.120361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:08.120390 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:08.123141 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.123852 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.123856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.123858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.123988 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.124361 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.124364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.124366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.124683 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:08.124792 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:08.125196 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 06:53:08.125358 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:53:08.125773 13825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:08.111731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50082)
0810 06:53:08.111931 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:08.111953 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:08.112085 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:08.112390 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:08.112394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:08.112923 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:08.113144 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:08.113156 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.114189 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.114195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:08.115713 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:08.115729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.116749 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.116756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:08.116955 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.117556 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:08.117578 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:08.120161 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:08.123266 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:08.123272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:08.123277 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:08.123751 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:08.124083 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:08.124088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:08.124091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:08.124264 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:08.124809 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:08.124814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:08.124979 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 06:53:08.125139 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 06:53:08.125405 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:53:08.131549 13823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9: Bootstrap starting.
I20260810 06:53:08.135535 13823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:08.136871 13823 log.cc:824] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:08.140054 13823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9: No bootstrap required, opened a new log
I20260810 06:53:08.147217 13823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.147511 13823 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:08.147630 13823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0682a48b7d444e98949293c1c1f749f9, State: Initialized, Role: FOLLOWER
I20260810 06:53:08.148180 13823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.149089 14147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.149509 14147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:53:08.150338 13823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:53:08.163381 14161 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:53:08.163506 14161 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:53:08.159721 13713 master_runner.cc:186] Error getting master registration for 127.22.11.189:46271: OK, instance_id { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" instance_seqno: 1786344787464690 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:53:08.168139 13713 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:53:08.181651 13959 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } attempt: 5
W20260810 06:53:08.182960 14014 tablet.cc:2369] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 06:53:08 dist-test-slave-f7th krb5kdc[13694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344787, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
I20260810 06:53:08.199038 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:08.182206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44571 (local address 127.0.0.1:50094)
0810 06:53:08.182446 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:08.182463 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:08.182578 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:08.183103 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:08.183108 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:08.183649 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:08.183864 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:08.183877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.186086 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.186091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:08.187498 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:08.187514 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.188504 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.188510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:08.188774 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.189439 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:08.189463 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:08.192193 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:08.196232 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:08.196238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:08.196242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:08.196802 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:08.197188 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:08.197194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:08.197198 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:08.197365 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:08.198026 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:08.198031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:08.198189 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 06:53:08.198347 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 06:53:08.198628 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260810 06:53:08.199122 14100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:08.182283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50094 (local address 127.22.11.188:44571)
0810 06:53:08.182449 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:08.182454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:08.182517 (+ 63us) server_negotiation.cc:407] Connection header received
0810 06:53:08.182711 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:08.182715 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:08.182791 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:08.182886 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:08.184926 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.185919 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.187667 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:08.188345 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:08.188480 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:08.192389 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:08.192396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:08.192401 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:08.192454 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:08.196091 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.196918 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.196924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.196928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.197071 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:08.197478 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:08.197482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:08.197486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:08.197823 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:08.197963 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:08.198488 (+ 525us) server_negotiation.cc:299] Negotiation successful
0810 06:53:08.198746 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:53:08.205307 13959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: Bootstrap starting.
I20260810 06:53:08.209486 13959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:08.210805 13959 log.cc:824] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:08.215432 13959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: No bootstrap required, opened a new log
W20260810 06:53:08.220818 13745 tablet.cc:2369] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:53:08.226059 13959 raft_consensus.cc:348] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.226451 13959 raft_consensus.cc:374] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:08.226573 13959 raft_consensus.cc:729] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da29e98e06e94fb690e59de497a5039a, State: Initialized, Role: FOLLOWER
I20260810 06:53:08.227303 13959 consensus_queue.cc:260] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.229043 14169 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.229478 14169 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:53:08.230317 13959 sys_catalog.cc:564] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:08.247352 13834 master_runner.cc:263] No leader master found from master da29e98e06e94fb690e59de497a5039a
I20260810 06:53:08.247491 13834 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 06:53:08.243615 14187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:53:08.248884 14187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:53:08.281919 14147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:53:08.282223 14147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.283864 14147 leader_election.cc:288] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da29e98e06e94fb690e59de497a5039a (127.22.11.189:46271), c2d001a875e9404c8fb5969a93c6f30c (127.22.11.188:44571)
I20260810 06:53:08.285544 13905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0682a48b7d444e98949293c1c1f749f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da29e98e06e94fb690e59de497a5039a" is_pre_election: true
I20260810 06:53:08.286158 13905 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0682a48b7d444e98949293c1c1f749f9 in term 0.
I20260810 06:53:08.287250 14050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0682a48b7d444e98949293c1c1f749f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d001a875e9404c8fb5969a93c6f30c" is_pre_election: true
I20260810 06:53:08.287833 14050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0682a48b7d444e98949293c1c1f749f9 in term 0.
I20260810 06:53:08.288365 13740 leader_election.cc:302] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0682a48b7d444e98949293c1c1f749f9, da29e98e06e94fb690e59de497a5039a; no voters:
I20260810 06:53:08.289312 14147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:53:08.289459 14147 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:53:08.289564 14147 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:08.296277 14147 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.299131 13905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0682a48b7d444e98949293c1c1f749f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da29e98e06e94fb690e59de497a5039a"
I20260810 06:53:08.299348 13905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:08.302991 13905 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0682a48b7d444e98949293c1c1f749f9 in term 1.
I20260810 06:53:08.303490 13740 leader_election.cc:302] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0682a48b7d444e98949293c1c1f749f9, da29e98e06e94fb690e59de497a5039a; no voters:
I20260810 06:53:08.306039 14050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0682a48b7d444e98949293c1c1f749f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2d001a875e9404c8fb5969a93c6f30c"
I20260810 06:53:08.306242 14050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:08.309612 14050 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0682a48b7d444e98949293c1c1f749f9 in term 1.
I20260810 06:53:08.310302 14147 leader_election.cc:288] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [CANDIDATE]: Term 1 election: Requested vote from peers da29e98e06e94fb690e59de497a5039a (127.22.11.189:46271), c2d001a875e9404c8fb5969a93c6f30c (127.22.11.188:44571)
I20260810 06:53:08.310914 14147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:08.311299 14147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [term 1 LEADER]: Becoming Leader. State: Replica: 0682a48b7d444e98949293c1c1f749f9, State: Running, Role: LEADER
I20260810 06:53:08.312098 14147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } }
I20260810 06:53:08.325032 14147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0682a48b7d444e98949293c1c1f749f9. Latest consensus state: current_term: 1 leader_uuid: "0682a48b7d444e98949293c1c1f749f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.327070 14147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:08.334249 14194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:53:08.336731 14194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:53:08.356024 14050 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [term 1 FOLLOWER]: Refusing update from remote peer 0682a48b7d444e98949293c1c1f749f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:08.357246 14147 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:08.364634 13905 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [term 1 FOLLOWER]: Refusing update from remote peer 0682a48b7d444e98949293c1c1f749f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:08.365590 14147 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:08.404753 14169 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0682a48b7d444e98949293c1c1f749f9. Latest consensus state: current_term: 1 leader_uuid: "0682a48b7d444e98949293c1c1f749f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.405108 14169 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:53:08.410511 14198 mvcc.cc:205] Tried to move back new op lower bound from 7316868253095510016 to 7316868252937887744. Current Snapshot: MvccSnapshot[applied={T|T < 7316868253095510016}]
I20260810 06:53:08.422422 14200 mvcc.cc:205] Tried to move back new op lower bound from 7316868253095510016 to 7316868252937887744. Current Snapshot: MvccSnapshot[applied={T|T < 7316868253095510016}]
I20260810 06:53:08.434924 14147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0682a48b7d444e98949293c1c1f749f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.435283 14147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:08.442991 14194 catalog_manager.cc:1347] Generated new cluster ID: 0f48baa25ce24310b6c0f2812d3fb46c
I20260810 06:53:08.443102 14194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:53:08.444132 14169 sys_catalog.cc:455] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0682a48b7d444e98949293c1c1f749f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.444382 14169 sys_catalog.cc:458] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:53:08.447019 14147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0682a48b7d444e98949293c1c1f749f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.447321 14147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9 [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:08.479403 14210 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0682a48b7d444e98949293c1c1f749f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.479902 14210 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:53:08.480537 14120 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0682a48b7d444e98949293c1c1f749f9. Latest consensus state: current_term: 1 leader_uuid: "0682a48b7d444e98949293c1c1f749f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0682a48b7d444e98949293c1c1f749f9" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34567 } } peers { permanent_uuid: "da29e98e06e94fb690e59de497a5039a" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 46271 } } peers { permanent_uuid: "c2d001a875e9404c8fb5969a93c6f30c" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44571 } } }
I20260810 06:53:08.480831 14120 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:53:08.484897 14194 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:53:08.487290 14194 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:53:08.502936 14194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0682a48b7d444e98949293c1c1f749f9: Generated new TSK 0
I20260810 06:53:08.503861 14194 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:53:09.065907 14138 catalog_manager.cc:1259] Loaded cluster ID: 0f48baa25ce24310b6c0f2812d3fb46c
I20260810 06:53:09.066015 14138 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: loading cluster ID for follower catalog manager: success
I20260810 06:53:09.068496 14138 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: acquiring CA information for follower catalog manager: success
I20260810 06:53:09.070322 14138 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2d001a875e9404c8fb5969a93c6f30c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:53:09.251353 14187 catalog_manager.cc:1259] Loaded cluster ID: 0f48baa25ce24310b6c0f2812d3fb46c
I20260810 06:53:09.251447 14187 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: loading cluster ID for follower catalog manager: success
I20260810 06:53:09.253880 14187 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: acquiring CA information for follower catalog manager: success
I20260810 06:53:09.255604 14187 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da29e98e06e94fb690e59de497a5039a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:53:09.610013 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 13713
I20260810 06:53:09.611305 13817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:09.723968 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 13713
I20260810 06:53:09.754164 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 13834
I20260810 06:53:09.755208 13949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:09.861711 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 13834
I20260810 06:53:09.888355 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 13972
I20260810 06:53:09.889443 14083 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:09.995793 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 13972
2026-08-10T06:53:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4929 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:53:10 dist-test-slave-f7th krb5kdc[14229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:53:10 dist-test-slave-f7th krb5kdc[14229](info): set up 2 sockets
Aug 10 06:53:10 dist-test-slave-f7th krb5kdc[14229](info): commencing operation
krb5kdc: starting...
W20260810 06:53:11.255218 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:53:11 dist-test-slave-f7th krb5kdc[14229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:53:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:53:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:53:11.492859 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37383
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:40999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:37383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:11.813019 14258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:11.813273 14258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:11.813367 14258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:11.820261 14258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:11.820353 14258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:11.820405 14258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:11.820461 14258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:11.820502 14258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:11.829960 14258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:40999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:37383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:11.831105 14258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:11.832394 14258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:11.843691 14271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:11.843731 14266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:11.843758 14268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:11.844767 14258 server_base.cc:1034] running on GCE node
I20260810 06:53:11.845566 14258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:11.846899 14258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:11.848117 14258 hybrid_clock.cc:648] HybridClock initialized: now 1786344791848058 us; error 42 us; skew 500 ppm
I20260810 06:53:11.848446 14258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:11 dist-test-slave-f7th krb5kdc[14229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344791, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:11.851378 14258 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:53:11.852985 14258 webserver.cc:460] Webserver started at http://127.22.11.190:41693/ using document root <none> and password file <none>
I20260810 06:53:11.853812 14258 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:11.853962 14258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:11.854230 14258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:11.857102 14258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "f6a867e3f4464d8cb12974e5cd6c9080"
format_stamp: "Formatted at 2026-08-10 06:53:11 on dist-test-slave-f7th"
server_key: "d27a277ca76d1abd9dda29180571ddc8"
server_key_iv: "fe62f9cf44c358931ac7ecc1e7223713"
server_key_version: "encryptionkey@0"
I20260810 06:53:11.857791 14258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "f6a867e3f4464d8cb12974e5cd6c9080"
format_stamp: "Formatted at 2026-08-10 06:53:11 on dist-test-slave-f7th"
server_key: "d27a277ca76d1abd9dda29180571ddc8"
server_key_iv: "fe62f9cf44c358931ac7ecc1e7223713"
server_key_version: "encryptionkey@0"
I20260810 06:53:11.862727 14258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:53:11.865890 14287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:11.867093 14258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:53:11.867250 14258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "f6a867e3f4464d8cb12974e5cd6c9080"
format_stamp: "Formatted at 2026-08-10 06:53:11 on dist-test-slave-f7th"
server_key: "d27a277ca76d1abd9dda29180571ddc8"
server_key_iv: "fe62f9cf44c358931ac7ecc1e7223713"
server_key_version: "encryptionkey@0"
I20260810 06:53:11.867417 14258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:11.890461 14258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:11.894084 14258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:11.894328 14258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:11.911243 14258 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:37383
I20260810 06:53:11.911247 14370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:37383 every 8 connection(s)
I20260810 06:53:11.912554 14258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:53:11.913604 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 14258
I20260810 06:53:11.913818 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:53:11.914463 22574 external_mini_cluster.cc:1428] Setting key f8500d568d473097b7f003322f5bf7e2
I20260810 06:53:11.916525 14374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:53:11 dist-test-slave-f7th krb5kdc[14229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:11.929024 14374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080: Bootstrap starting.
I20260810 06:53:11.931993 14374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:11.933032 14374 log.cc:824] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:11.935678 14374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080: No bootstrap required, opened a new log
I20260810 06:53:11.936329 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:11.916203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.22.11.190:37383)
0810 06:53:11.917015 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:11.917026 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:11.917065 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:53:11.918651 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:11.918671 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:11.919028 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:11.919349 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:11.921148 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:11.922445 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:11.924210 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:11.924874 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:11.925024 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:11.929577 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:11.929584 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:11.929597 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:11.929630 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:11.932472 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:11.933163 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:11.933172 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:11.933179 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:11.933348 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:11.933727 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:11.933730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:11.933732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:11.934262 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:11.934546 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:11.935004 (+ 458us) server_negotiation.cc:299] Negotiation successful
0810 06:53:11.935374 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":374,"threads_started":1}
I20260810 06:53:11.939827 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:53:11.943236 14374 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37383 } }
I20260810 06:53:11.943547 14374 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:11.943634 14374 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6a867e3f4464d8cb12974e5cd6c9080, State: Initialized, Role: FOLLOWER
I20260810 06:53:11.944166 14374 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37383 } }
I20260810 06:53:11.944387 14374 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:53:11.944509 14374 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:53:11.944700 14374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:11.946695 14374 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37383 } }
I20260810 06:53:11.947160 14374 leader_election.cc:302] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6a867e3f4464d8cb12974e5cd6c9080; no voters:
I20260810 06:53:11.947690 14374 leader_election.cc:288] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:53:11.947800 14383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:11.948403 14383 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [term 1 LEADER]: Becoming Leader. State: Replica: f6a867e3f4464d8cb12974e5cd6c9080, State: Running, Role: LEADER
I20260810 06:53:11.948979 14383 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37383 } }
I20260810 06:53:11.949556 14374 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:11.951527 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37383 } } }
I20260810 06:53:11.951920 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:11.951859 14388 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6a867e3f4464d8cb12974e5cd6c9080. Latest consensus state: current_term: 1 leader_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6a867e3f4464d8cb12974e5cd6c9080" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37383 } } }
I20260810 06:53:11.952159 14388 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080 [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:11.953670 14397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:53:11.958503 14397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:53:11.966457 14397 catalog_manager.cc:1347] Generated new cluster ID: a478769990d94f2b8184aca67ba858b7
I20260810 06:53:11.966570 14397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:53:11.975780 14397 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:53:11.977516 14397 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:53:11.992172 14397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6a867e3f4464d8cb12974e5cd6c9080: Generated new TSK 0
I20260810 06:53:11.993130 14397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:53:12.050884 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 14258
I20260810 06:53:12.052001 14364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:12.149212 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 14258
2026-08-10T06:53:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2149 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:53:12 dist-test-slave-f7th krb5kdc[14430](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:53:12 dist-test-slave-f7th krb5kdc[14430](info): set up 2 sockets
Aug 10 06:53:12 dist-test-slave-f7th krb5kdc[14430](info): commencing operation
krb5kdc: starting...
W20260810 06:53:13.394860 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:53:13 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:53:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:53:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:53:13.627640 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43427
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:13.949327 14454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:13.949587 14454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:13.949684 14454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:13.956607 14454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:13.956720 14454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:13.956779 14454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:13.956835 14454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:13.956884 14454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:13.966179 14454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:43427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:13.967319 14454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:13.968502 14454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:13.979511 14461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:13.979557 14467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:13.979590 14462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:13.980698 14454 server_base.cc:1034] running on GCE node
I20260810 06:53:13.981380 14454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:13.982631 14454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:13.983829 14454 hybrid_clock.cc:648] HybridClock initialized: now 1786344793983788 us; error 27 us; skew 500 ppm
I20260810 06:53:13.984125 14454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:13 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344793, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:13.990996 14454 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:53:13.992367 14454 webserver.cc:460] Webserver started at http://127.22.11.190:43861/ using document root <none> and password file <none>
I20260810 06:53:13.993053 14454 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:13.993160 14454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:13.993407 14454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:13.996191 14454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "8826c08e00bf40c79ba1d9c0d465dccd"
format_stamp: "Formatted at 2026-08-10 06:53:13 on dist-test-slave-f7th"
server_key: "5024bbd9a7b65cb133ec51de8623d515"
server_key_iv: "cefcfb9e81a24f0aac42f5e95bba3ada"
server_key_version: "encryptionkey@0"
I20260810 06:53:13.996966 14454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "8826c08e00bf40c79ba1d9c0d465dccd"
format_stamp: "Formatted at 2026-08-10 06:53:13 on dist-test-slave-f7th"
server_key: "5024bbd9a7b65cb133ec51de8623d515"
server_key_iv: "cefcfb9e81a24f0aac42f5e95bba3ada"
server_key_version: "encryptionkey@0"
I20260810 06:53:14.001778 14454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:53:14.005043 14480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:14.006318 14454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:53:14.006496 14454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "8826c08e00bf40c79ba1d9c0d465dccd"
format_stamp: "Formatted at 2026-08-10 06:53:13 on dist-test-slave-f7th"
server_key: "5024bbd9a7b65cb133ec51de8623d515"
server_key_iv: "cefcfb9e81a24f0aac42f5e95bba3ada"
server_key_version: "encryptionkey@0"
I20260810 06:53:14.006666 14454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:14.041631 14454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:14.045671 14454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:14.045920 14454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:14.063217 14454 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:43427
I20260810 06:53:14.063256 14556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:43427 every 8 connection(s)
I20260810 06:53:14.064504 14454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:53:14.068040 14561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:14.069543 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 14454
I20260810 06:53:14.069739 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:53:14.070410 22574 external_mini_cluster.cc:1428] Setting key 7a0e91f38d9c769b19c67bf4ac09ff3f
I20260810 06:53:14.079417 14561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Bootstrap starting.
I20260810 06:53:14.083001 14561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:14.084252 14561 log.cc:824] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:53:14 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:14.087181 14561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: No bootstrap required, opened a new log
I20260810 06:53:14.093147 14565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:14.072478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47906 (local address 127.22.11.190:43427)
0810 06:53:14.073520 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:14.073531 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:14.073570 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:53:14.075187 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:14.075209 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:14.075559 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:14.075862 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:14.077587 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:14.078833 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:14.080604 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:14.081290 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:14.081431 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:14.086224 (+ 4793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:14.086231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:14.086244 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:14.086276 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:14.089240 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:14.090039 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:14.090045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:14.090050 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:14.090207 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:14.090587 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:14.090590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:14.090592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:14.091099 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:14.091337 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:14.091774 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 06:53:14.092114 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":564,"threads_started":1}
I20260810 06:53:14.095774 14561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:14.096135 14561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:14.096222 14561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8826c08e00bf40c79ba1d9c0d465dccd, State: Initialized, Role: FOLLOWER
I20260810 06:53:14.096771 14561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:14.096982 14561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:53:14.097121 14561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:53:14.097271 14561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:14.099246 14561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:14.099684 14561 leader_election.cc:302] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8826c08e00bf40c79ba1d9c0d465dccd; no voters:
I20260810 06:53:14.100200 14561 leader_election.cc:288] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:53:14.100309 14571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:14.101001 14571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 LEADER]: Becoming Leader. State: Replica: 8826c08e00bf40c79ba1d9c0d465dccd, State: Running, Role: LEADER
I20260810 06:53:14.101552 14571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:14.102269 14561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:14.104885 14572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } } }
I20260810 06:53:14.105708 14572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:14.104979 14573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8826c08e00bf40c79ba1d9c0d465dccd. Latest consensus state: current_term: 1 leader_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } } }
I20260810 06:53:14.106318 14573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:14.108378 14585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:53:14.112772 14585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:53:14.121073 14585 catalog_manager.cc:1347] Generated new cluster ID: a209bb5335e04b4ba6a3e6962b78fb81
I20260810 06:53:14.121198 14585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:53:14.145949 14585 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:53:14.147243 14585 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:53:14.153635 14585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Generated new TSK 0
I20260810 06:53:14.154407 14585 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:53:14.227802 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:43427
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:14.548439 14610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:14.548714 14610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:14.548842 14610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:14.555706 14610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:14.555800 14610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:14.555939 14610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:53:14.565052 14610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:14.566161 14610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:14.567413 14610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:14.579521 14619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:14.579478 14620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:14.579559 14623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:14.580531 14610 server_base.cc:1034] running on GCE node
I20260810 06:53:14.581133 14610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:14.581971 14610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:14.583173 14610 hybrid_clock.cc:648] HybridClock initialized: now 1786344794583119 us; error 46 us; skew 500 ppm
I20260810 06:53:14.583455 14610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:14 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344794, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:14.586421 14610 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:53:14.587666 14610 webserver.cc:460] Webserver started at http://127.22.11.129:42457/ using document root <none> and password file <none>
I20260810 06:53:14.588259 14610 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:14.588359 14610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:14.588595 14610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:14.591449 14610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "d62edb8e04b2464383c438609ad4213e"
format_stamp: "Formatted at 2026-08-10 06:53:14 on dist-test-slave-f7th"
server_key: "5d67f73707987521e0cfa2c2fa2f15e1"
server_key_iv: "a6fce0bfbc13d6c36e529ce105eb2b8c"
server_key_version: "encryptionkey@0"
I20260810 06:53:14.592252 14610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "d62edb8e04b2464383c438609ad4213e"
format_stamp: "Formatted at 2026-08-10 06:53:14 on dist-test-slave-f7th"
server_key: "5d67f73707987521e0cfa2c2fa2f15e1"
server_key_iv: "a6fce0bfbc13d6c36e529ce105eb2b8c"
server_key_version: "encryptionkey@0"
I20260810 06:53:14.597051 14610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 06:53:14.600265 14640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:14.601637 14610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:53:14.601780 14610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "d62edb8e04b2464383c438609ad4213e"
format_stamp: "Formatted at 2026-08-10 06:53:14 on dist-test-slave-f7th"
server_key: "5d67f73707987521e0cfa2c2fa2f15e1"
server_key_iv: "a6fce0bfbc13d6c36e529ce105eb2b8c"
server_key_version: "encryptionkey@0"
I20260810 06:53:14.601951 14610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:14.617798 14610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:14.621631 14610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:14.621893 14610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:14.622721 14610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:14.623982 14610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:14.624059 14610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:14.624152 14610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:14.624194 14610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:14.654281 14610 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:33435
I20260810 06:53:14.654337 14832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:33435 every 8 connection(s)
I20260810 06:53:14.655483 14610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:53:14.661284 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 14610
I20260810 06:53:14.661484 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:53:14.662113 22574 external_mini_cluster.cc:1428] Setting key 774ddd1d2db25f0bcae588e8d0053fcb
Aug 10 06:53:14 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344794, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:14.674538 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:14.657834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:56107 (local address 127.22.11.190:43427)
0810 06:53:14.658266 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:14.658271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:14.659198 (+ 927us) server_negotiation.cc:407] Connection header received
0810 06:53:14.660937 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:14.660944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:14.661030 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:14.661160 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:14.662622 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:14.663455 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:14.665119 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:14.665682 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:14.665811 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:14.668979 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:14.668986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:14.668990 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:14.669029 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:14.671805 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:14.672441 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:14.672447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:14.672450 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:14.672592 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:14.672974 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:14.672978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:14.672981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:14.673294 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:14.673427 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:14.674004 (+ 577us) server_negotiation.cc:299] Negotiation successful
0810 06:53:14.674220 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260810 06:53:14.675179 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:14.658367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:56107)
0810 06:53:14.659047 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:14.659091 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:14.660671 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:14.661327 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:14.661334 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:14.661863 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:14.662453 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:14.662470 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:14.663618 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:14.663624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:14.664974 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:14.664987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:14.665862 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:14.665866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:14.666055 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:14.666665 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:14.666692 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:14.668779 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:14.671924 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:14.671930 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:14.671941 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:14.672321 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:14.672697 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:14.672699 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:14.672702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:14.672874 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:14.673389 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:14.673394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:14.673668 (+ 274us) client_negotiation.cc:769] Sending connection context
0810 06:53:14.673866 (+ 198us) client_negotiation.cc:240] Negotiation successful
0810 06:53:14.674193 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":258,"threads_started":1}
I20260810 06:53:14.676709 14833 heartbeater.cc:347] Connected to a master server at 127.22.11.190:43427
I20260810 06:53:14.676976 14833 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:14.677668 14833 heartbeater.cc:511] Master 127.22.11.190:43427 requested a full tablet report, sending...
I20260810 06:53:14.679612 14504 ts_manager.cc:194] Registered new tserver with Master: d62edb8e04b2464383c438609ad4213e (127.22.11.129:33435)
I20260810 06:53:14.681068 14504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:56107
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:53:14.717298 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:43427
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:15.038277 14839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:15.038547 14839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:15.038684 14839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:15.046109 14839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:15.046219 14839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:15.046370 14839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:53:15.055481 14839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:15.056641 14839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:15.057883 14839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:15.071125 14852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:15.071105 14854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:15.075695 14847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:15.076965 14839 server_base.cc:1034] running on GCE node
I20260810 06:53:15.077805 14839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:15.079907 14839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:15.081207 14839 hybrid_clock.cc:648] HybridClock initialized: now 1786344795081095 us; error 91 us; skew 500 ppm
I20260810 06:53:15.081583 14839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:15 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344795, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:15.084924 14839 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:53:15.086259 14839 webserver.cc:460] Webserver started at http://127.22.11.130:44865/ using document root <none> and password file <none>
I20260810 06:53:15.086947 14839 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:15.087162 14839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:15.087484 14839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:15.090325 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "71e18bbe13084a968c7384a158972291"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "52f6937dc7a811e62a8d1ce1ec6aa359"
server_key_iv: "a5185953ae1bdae05154460d329290e5"
server_key_version: "encryptionkey@0"
I20260810 06:53:15.091080 14839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "71e18bbe13084a968c7384a158972291"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "52f6937dc7a811e62a8d1ce1ec6aa359"
server_key_iv: "a5185953ae1bdae05154460d329290e5"
server_key_version: "encryptionkey@0"
I20260810 06:53:15.095889 14839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:53:15.099249 14866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:15.100476 14839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:53:15.100694 14839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "71e18bbe13084a968c7384a158972291"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "52f6937dc7a811e62a8d1ce1ec6aa359"
server_key_iv: "a5185953ae1bdae05154460d329290e5"
server_key_version: "encryptionkey@0"
I20260810 06:53:15.100867 14839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:15.136735 14839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:15.140978 14839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:15.141258 14839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:15.142231 14839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:15.143561 14839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:15.143642 14839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:15.143776 14839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:15.143832 14839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:15.173640 14839 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35379
I20260810 06:53:15.173692 15085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35379 every 8 connection(s)
I20260810 06:53:15.174882 14839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:53:15.178740 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 14839
I20260810 06:53:15.178944 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:53:15.179414 22574 external_mini_cluster.cc:1428] Setting key 78dcb957ed823bcc00a736cbc6408973
Aug 10 06:53:15 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344795, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:15.193835 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.177496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:51521 (local address 127.22.11.190:43427)
0810 06:53:15.177929 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:15.177934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:15.178789 (+ 855us) server_negotiation.cc:407] Connection header received
0810 06:53:15.180731 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:15.180737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:15.180818 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:15.180913 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:15.182391 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.183110 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.184593 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.185033 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.185148 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.188389 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:15.188394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:15.188397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:15.188428 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:15.190871 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:15.191516 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:15.191522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:15.191525 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:15.191653 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:15.192099 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:15.192103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:15.192105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:15.192403 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:15.192547 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:15.193341 (+ 794us) server_negotiation.cc:299] Negotiation successful
0810 06:53:15.193509 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":223,"threads_started":1}
I20260810 06:53:15.195117 15093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.177962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:51521)
0810 06:53:15.178642 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:15.178680 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:15.180455 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:15.181180 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:15.181193 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:15.181692 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:15.182236 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.182254 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.183253 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.183257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.184460 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.184473 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.185253 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.185260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.185462 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.186057 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:15.186083 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:15.188185 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:15.190993 (+ 2808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:15.191002 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:15.191016 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:15.191407 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:15.191752 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:15.191757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:15.191761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:15.191976 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:15.192543 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:15.192552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:15.192980 (+ 428us) client_negotiation.cc:769] Sending connection context
0810 06:53:15.193282 (+ 302us) client_negotiation.cc:240] Negotiation successful
0810 06:53:15.193766 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":258,"threads_started":1}
I20260810 06:53:15.196511 15088 heartbeater.cc:347] Connected to a master server at 127.22.11.190:43427
I20260810 06:53:15.196869 15088 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:15.197705 15088 heartbeater.cc:511] Master 127.22.11.190:43427 requested a full tablet report, sending...
I20260810 06:53:15.199190 14504 ts_manager.cc:194] Registered new tserver with Master: 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379)
I20260810 06:53:15.199990 14504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:51521
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:53:15.239079 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:43427
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:15.556813 15098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:15.557097 15098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:15.557240 15098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:15.564098 15098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:15.564205 15098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:15.564357 15098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:53:15.573513 15098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:15.574676 15098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:15.575887 15098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:15.588002 15105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:15.588039 15110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:15.588063 15104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:15.588936 15098 server_base.cc:1034] running on GCE node
I20260810 06:53:15.589452 15098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:15.590235 15098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:15.591416 15098 hybrid_clock.cc:648] HybridClock initialized: now 1786344795591367 us; error 42 us; skew 500 ppm
I20260810 06:53:15.591673 15098 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:15 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344795, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:15.594623 15098 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:53:15.595959 15098 webserver.cc:460] Webserver started at http://127.22.11.131:41345/ using document root <none> and password file <none>
I20260810 06:53:15.596562 15098 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:15.596701 15098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:15.596987 15098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:15.599875 15098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "92d85f473aee4d828e604a0d4582309d"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "84bdf50704dcb45f4968ee857ed9fb19"
server_key_iv: "9d3b5df5f4dcd9f6e0cb933b914526a4"
server_key_version: "encryptionkey@0"
I20260810 06:53:15.600664 15098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "92d85f473aee4d828e604a0d4582309d"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "84bdf50704dcb45f4968ee857ed9fb19"
server_key_iv: "9d3b5df5f4dcd9f6e0cb933b914526a4"
server_key_version: "encryptionkey@0"
I20260810 06:53:15.605564 15098 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:53:15.609037 15124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:15.610276 15098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:53:15.610422 15098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "92d85f473aee4d828e604a0d4582309d"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "84bdf50704dcb45f4968ee857ed9fb19"
server_key_iv: "9d3b5df5f4dcd9f6e0cb933b914526a4"
server_key_version: "encryptionkey@0"
I20260810 06:53:15.610600 15098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:15.640216 15098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:15.644308 15098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:15.644584 15098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:15.645555 15098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:15.646886 15098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:15.646968 15098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:15.647092 15098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:15.647156 15098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:15.676492 15098 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:42281
I20260810 06:53:15.676543 15329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:42281 every 8 connection(s)
I20260810 06:53:15.677743 15098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:53:15.682194 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 15098
I20260810 06:53:15.682360 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:53:15.682780 22574 external_mini_cluster.cc:1428] Setting key ae97df2d2ef69e756342c4af54f3d133
I20260810 06:53:15.683713 14833 heartbeater.cc:503] Master 127.22.11.190:43427 was elected leader, sending a full tablet report...
Aug 10 06:53:15 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344795, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:15.696265 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.680068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:58149 (local address 127.22.11.190:43427)
0810 06:53:15.680233 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:15.680238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:15.681360 (+ 1122us) server_negotiation.cc:407] Connection header received
0810 06:53:15.683140 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:15.683146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:15.683237 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:15.683347 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:15.685219 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.686068 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.687563 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.687976 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.688071 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.691248 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:15.691252 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:15.691256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:15.691285 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:15.693525 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:15.694150 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:15.694153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:15.694156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:15.694260 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:15.694620 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:15.694623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:15.694625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:15.694866 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:15.694996 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:15.695885 (+ 889us) server_negotiation.cc:299] Negotiation successful
0810 06:53:15.696049 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:53:15.697090 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.680486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:58149)
0810 06:53:15.681186 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:15.681222 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:15.682853 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:15.683613 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:15.683621 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:15.684261 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:15.685023 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.685049 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.686234 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.686237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.687420 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.687433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.688139 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.688143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.688322 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.688923 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:15.688944 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:15.691073 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:15.693662 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:15.693669 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:15.693680 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:15.694063 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:15.694353 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:15.694357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:15.694359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:15.694539 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:15.695014 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:15.695020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:15.695312 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 06:53:15.695548 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 06:53:15.695897 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":305,"threads_started":1}
I20260810 06:53:15.698360 15332 heartbeater.cc:347] Connected to a master server at 127.22.11.190:43427
I20260810 06:53:15.698619 15332 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:15.699275 15332 heartbeater.cc:511] Master 127.22.11.190:43427 requested a full tablet report, sending...
I20260810 06:53:15.700603 14501 ts_manager.cc:194] Registered new tserver with Master: 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281)
I20260810 06:53:15.701434 14501 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:58149
I20260810 06:53:15.710150 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:53:15.726486 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.714510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.22.11.190:43427)
0810 06:53:15.714652 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:15.714655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:15.715111 (+ 456us) server_negotiation.cc:407] Connection header received
0810 06:53:15.715313 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:15.715317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:15.715375 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:15.715451 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:15.716884 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.717608 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.718909 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.719335 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.719475 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.721741 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:15.721746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:15.721749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:15.721776 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:15.724027 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:15.724650 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:15.724656 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:15.724658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:15.724769 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:15.725134 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:15.725140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:15.725143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:15.725435 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:15.725683 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:15.725974 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 06:53:15.726181 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:53:15.730146 14501 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:53:15.736799 14501 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:53:15.768281 15367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.748718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56296 (local address 127.22.11.129:33435)
0810 06:53:15.760969 (+ 12251us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:15.760977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:15.761001 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:53:15.761104 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:15.761111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:15.761323 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:15.761508 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:15.762157 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.763302 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.765768 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.767167 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.767304 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.767536 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:15.767792 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 06:53:15.767949 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12077,"thread_start_us":11911,"threads_started":1}
I20260810 06:53:15.768393 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.748287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:33435 (local address 127.0.0.1:56296)
0810 06:53:15.759512 (+ 11225us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:15.759545 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:15.759645 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:15.761510 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:15.761514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:15.761548 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:15.761942 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.761957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.763504 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.763510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.765619 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.765630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.767377 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.767383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.767560 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.767609 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 06:53:15.767727 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:53:15.767910 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11106,"mutex_wait_us":78,"thread_start_us":9724,"threads_started":1}
I20260810 06:53:15.768551 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.747926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:42281 (local address 127.0.0.1:50684)
0810 06:53:15.758916 (+ 10990us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:15.758938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:15.759041 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:15.761142 (+ 2101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:15.761147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:15.761180 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:15.761802 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.761827 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.763715 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.763721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.766046 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.766059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.767697 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.767704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.767863 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.767894 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:53:15.767972 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 06:53:15.768112 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10867,"mutex_wait_us":58,"thread_start_us":11312,"threads_started":1}
I20260810 06:53:15.769156 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.747726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35379 (local address 127.0.0.1:53408)
0810 06:53:15.757627 (+ 9901us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:15.757680 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:15.757845 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:15.760801 (+ 2956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:15.760808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:15.760848 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:15.761249 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.761262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.763393 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.763398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.766312 (+ 2914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:15.766325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.768330 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.768339 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:15.768551 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.768583 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:53:15.768703 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:53:15.768821 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9643,"mutex_wait_us":132,"thread_start_us":10939,"threads_started":1}
I20260810 06:53:15.769495 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.748611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.22.11.131:42281)
0810 06:53:15.760650 (+ 12039us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:15.760658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:15.760688 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:53:15.760784 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:15.760791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:15.760962 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:15.761142 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:15.762091 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.763579 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.766218 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.767188 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.767306 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.767551 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:15.768960 (+ 1409us) server_negotiation.cc:299] Negotiation successful
0810 06:53:15.769134 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11801,"thread_start_us":11616,"threads_started":1}
I20260810 06:53:15.769956 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:15.747989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.22.11.130:35379)
0810 06:53:15.760133 (+ 12144us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:15.760143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:15.760173 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:53:15.760274 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:15.760281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:15.760547 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:15.761467 (+ 920us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:15.761712 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.763257 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.766465 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:15.767863 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:15.767992 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:15.768164 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:15.769472 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0810 06:53:15.769650 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11963,"thread_start_us":11862,"threads_started":1}
I20260810 06:53:15.772706 14717 tablet_service.cc:1467] Processing CreateTablet for tablet db88c85d939d4e13a2522110e5092d45 (DEFAULT_TABLE table=test-table [id=ceaf532047ac4c8cbd9d3827c09d1b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:15.772957 14960 tablet_service.cc:1467] Processing CreateTablet for tablet db88c85d939d4e13a2522110e5092d45 (DEFAULT_TABLE table=test-table [id=ceaf532047ac4c8cbd9d3827c09d1b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:15.773088 15216 tablet_service.cc:1467] Processing CreateTablet for tablet db88c85d939d4e13a2522110e5092d45 (DEFAULT_TABLE table=test-table [id=ceaf532047ac4c8cbd9d3827c09d1b03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:15.774075 14960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db88c85d939d4e13a2522110e5092d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:15.774209 14717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db88c85d939d4e13a2522110e5092d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:15.774279 15216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db88c85d939d4e13a2522110e5092d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:15.785537 15368 tablet_bootstrap.cc:492] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Bootstrap starting.
I20260810 06:53:15.788851 15368 tablet_bootstrap.cc:654] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:15.789915 15368 log.cc:824] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:15.792547 15368 tablet_bootstrap.cc:492] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: No bootstrap required, opened a new log
I20260810 06:53:15.792804 15368 ts_tablet_manager.cc:1397] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260810 06:53:15.792879 15371 tablet_bootstrap.cc:492] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Bootstrap starting.
I20260810 06:53:15.793781 15369 tablet_bootstrap.cc:492] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Bootstrap starting.
I20260810 06:53:15.796360 15371 tablet_bootstrap.cc:654] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:15.797115 15369 tablet_bootstrap.cc:654] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:15.797768 15371 log.cc:824] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:15.798394 15369 log.cc:824] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:15.800374 15371 tablet_bootstrap.cc:492] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: No bootstrap required, opened a new log
I20260810 06:53:15.800673 15371 ts_tablet_manager.cc:1397] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 06:53:15.800884 15368 raft_consensus.cc:348] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:15.801323 15368 raft_consensus.cc:374] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:15.801430 15368 raft_consensus.cc:729] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e18bbe13084a968c7384a158972291, State: Initialized, Role: FOLLOWER
I20260810 06:53:15.802017 15368 consensus_queue.cc:260] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:15.802863 15369 tablet_bootstrap.cc:492] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: No bootstrap required, opened a new log
I20260810 06:53:15.803129 15369 ts_tablet_manager.cc:1397] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 06:53:15.805459 15088 heartbeater.cc:503] Master 127.22.11.190:43427 was elected leader, sending a full tablet report...
I20260810 06:53:15.805955 15368 ts_tablet_manager.cc:1428] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260810 06:53:15.811125 15371 raft_consensus.cc:348] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:15.811493 15371 raft_consensus.cc:374] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:15.811596 15371 raft_consensus.cc:729] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d85f473aee4d828e604a0d4582309d, State: Initialized, Role: FOLLOWER
I20260810 06:53:15.812302 15371 consensus_queue.cc:260] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:15.812956 15369 raft_consensus.cc:348] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:15.813426 15332 heartbeater.cc:503] Master 127.22.11.190:43427 was elected leader, sending a full tablet report...
I20260810 06:53:15.813470 15369 raft_consensus.cc:374] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:15.813557 15369 raft_consensus.cc:729] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62edb8e04b2464383c438609ad4213e, State: Initialized, Role: FOLLOWER
I20260810 06:53:15.813898 15371 ts_tablet_manager.cc:1428] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Time spent starting tablet: real 0.013s user 0.007s sys 0.004s
I20260810 06:53:15.814158 15369 consensus_queue.cc:260] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:15.815837 15369 ts_tablet_manager.cc:1428] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Time spent starting tablet: real 0.013s user 0.002s sys 0.009s
W20260810 06:53:15.907618 14834 tablet.cc:2369] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:53:15.926909 15089 tablet.cc:2369] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:53:15.929211 15333 tablet.cc:2369] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:53:16.034845 15379 raft_consensus.cc:482] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:53:16.035172 15379 raft_consensus.cc:504] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:16.037003 15379 leader_election.cc:288] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379), 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281)
I20260810 06:53:16.043598 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.037521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:46971 (local address 127.22.11.131:42281)
0810 06:53:16.037717 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:16.037724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:16.038475 (+ 751us) server_negotiation.cc:407] Connection header received
0810 06:53:16.038624 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:16.038628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:16.038691 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:16.038784 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:16.039405 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.040120 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.041951 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.042696 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.042837 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.042911 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:16.043231 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 06:53:16.043359 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:53:16.043632 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.037370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:60317 (local address 127.22.11.130:35379)
0810 06:53:16.037608 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:16.037615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:16.038376 (+ 761us) server_negotiation.cc:407] Connection header received
0810 06:53:16.038555 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:16.038561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:16.038638 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:16.038706 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:16.039347 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.040092 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.041968 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.042740 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.042889 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.042960 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:16.043254 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 06:53:16.043375 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:53:16.043737 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.037491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:42281 (local address 127.22.11.129:46971)
0810 06:53:16.038349 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:16.038367 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:16.038499 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:16.038809 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:16.038813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:16.038837 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:16.039232 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:16.039242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.040251 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.040254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:16.041794 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:16.041809 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.042910 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.042917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:16.043073 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.043101 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 06:53:16.043172 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:53:16.043285 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":20,"thread_start_us":427,"threads_started":1}
I20260810 06:53:16.044098 15381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.037407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35379 (local address 127.22.11.129:60317)
0810 06:53:16.038258 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:16.038274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:16.038403 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:16.038749 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:16.038754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:16.038786 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:16.039198 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:16.039208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.040223 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.040228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:16.041815 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:16.041827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.042972 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.042977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:16.043100 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.043122 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 06:53:16.043205 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:53:16.043323 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":91,"thread_start_us":490,"threads_started":1}
I20260810 06:53:16.044742 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db88c85d939d4e13a2522110e5092d45" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d85f473aee4d828e604a0d4582309d" is_pre_election: true
I20260810 06:53:16.044935 15002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db88c85d939d4e13a2522110e5092d45" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e18bbe13084a968c7384a158972291" is_pre_election: true
I20260810 06:53:16.045228 15256 raft_consensus.cc:2454] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 0.
I20260810 06:53:16.045409 15002 raft_consensus.cc:2454] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 0.
I20260810 06:53:16.045814 14645 leader_election.cc:302] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d85f473aee4d828e604a0d4582309d, d62edb8e04b2464383c438609ad4213e; no voters:
I20260810 06:53:16.046224 15379 raft_consensus.cc:2781] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:53:16.046350 15379 raft_consensus.cc:482] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:53:16.046440 15379 raft_consensus.cc:3037] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:16.048717 15379 raft_consensus.cc:504] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:16.049290 15379 leader_election.cc:288] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 election: Requested vote from peers 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379), 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281)
I20260810 06:53:16.049669 15002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db88c85d939d4e13a2522110e5092d45" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e18bbe13084a968c7384a158972291"
I20260810 06:53:16.049698 15256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db88c85d939d4e13a2522110e5092d45" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d85f473aee4d828e604a0d4582309d"
I20260810 06:53:16.049834 15002 raft_consensus.cc:3037] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:16.049854 15256 raft_consensus.cc:3037] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:16.051890 15002 raft_consensus.cc:2454] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 1.
I20260810 06:53:16.052047 15256 raft_consensus.cc:2454] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 1.
I20260810 06:53:16.052299 14642 leader_election.cc:302] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71e18bbe13084a968c7384a158972291, d62edb8e04b2464383c438609ad4213e; no voters:
I20260810 06:53:16.052591 15379 raft_consensus.cc:2781] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:16.052994 15379 raft_consensus.cc:686] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 1 LEADER]: Becoming Leader. State: Replica: d62edb8e04b2464383c438609ad4213e, State: Running, Role: LEADER
I20260810 06:53:16.053406 15379 consensus_queue.cc:237] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } }
I20260810 06:53:16.057473 14504 catalog_manager.cc:5495] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e reported cstate change: term changed from 0 to 1, leader changed from <none> to d62edb8e04b2464383c438609ad4213e (127.22.11.129). New cstate: current_term: 1 leader_uuid: "d62edb8e04b2464383c438609ad4213e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } health_report { overall_health: HEALTHY } } }
I20260810 06:53:16.131755 15367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.125954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56304 (local address 127.22.11.129:33435)
0810 06:53:16.126162 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:16.126168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:16.126188 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:53:16.126367 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:16.126371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:16.126496 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:16.126633 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:53:16.127087 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.128043 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.129893 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.130298 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.130389 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.130821 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:53:16.131311 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:53:16.131415 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:16.131455 (+ 40us) server_negotiation.cc:299] Negotiation successful
0810 06:53:16.131516 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:53:16.139621 15256 raft_consensus.cc:1261] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Refusing update from remote peer d62edb8e04b2464383c438609ad4213e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:16.139621 15002 raft_consensus.cc:1261] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Refusing update from remote peer d62edb8e04b2464383c438609ad4213e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:16.140686 15384 consensus_queue.cc:1035] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:16.141021 15379 consensus_queue.cc:1035] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:16.151546 15391 mvcc.cc:205] Tried to move back new op lower bound from 7316868284978151424 to 7316868284637990912. Current Snapshot: MvccSnapshot[applied={T|T < 7316868284978151424}]
I20260810 06:53:16.165068 14501 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:16.165360 14501 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:16.168937 14501 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Sending DeleteTablet for 3 replicas of tablet db88c85d939d4e13a2522110e5092d45
I20260810 06:53:16.170046 15216 tablet_service.cc:1514] Processing DeleteTablet for tablet db88c85d939d4e13a2522110e5092d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:16 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50684
I20260810 06:53:16.169999 14960 tablet_service.cc:1514] Processing DeleteTablet for tablet db88c85d939d4e13a2522110e5092d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:16 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:53408
I20260810 06:53:16.170202 14717 tablet_service.cc:1514] Processing DeleteTablet for tablet db88c85d939d4e13a2522110e5092d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:16 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:56296
I20260810 06:53:16.171778 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 14610
I20260810 06:53:16.183372 15411 tablet_replica.cc:330] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: stopping tablet replica
I20260810 06:53:16.183789 15411 raft_consensus.cc:2229] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:16.184289 15411 raft_consensus.cc:2258] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:16.187764 15411 ts_tablet_manager.cc:1905] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:16.191071 15407 tablet_replica.cc:330] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: stopping tablet replica
I20260810 06:53:16.191453 15407 raft_consensus.cc:2229] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:16.191943 15407 raft_consensus.cc:2258] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:16.195451 15407 ts_tablet_manager.cc:1905] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:16.196570 15412 tablet_replica.cc:330] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: stopping tablet replica
I20260810 06:53:16.197224 15412 raft_consensus.cc:2229] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:53:16.199693 15411 ts_tablet_manager.cc:1918] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:16.199867 15411 log.cc:1196] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/db88c85d939d4e13a2522110e5092d45
I20260810 06:53:16.200379 15411 ts_tablet_manager.cc:1939] T db88c85d939d4e13a2522110e5092d45 P 92d85f473aee4d828e604a0d4582309d: Deleting consensus metadata
I20260810 06:53:16.202253 15407 ts_tablet_manager.cc:1918] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:16.202301 14485 catalog_manager.cc:4841] TS 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281): tablet db88c85d939d4e13a2522110e5092d45 (table test-table [id=ceaf532047ac4c8cbd9d3827c09d1b03]) successfully deleted
I20260810 06:53:16.202400 15407 log.cc:1196] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/db88c85d939d4e13a2522110e5092d45
I20260810 06:53:16.202589 15412 raft_consensus.cc:2258] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:16.202937 15407 ts_tablet_manager.cc:1939] T db88c85d939d4e13a2522110e5092d45 P 71e18bbe13084a968c7384a158972291: Deleting consensus metadata
I20260810 06:53:16.204495 14483 catalog_manager.cc:4841] TS 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379): tablet db88c85d939d4e13a2522110e5092d45 (table test-table [id=ceaf532047ac4c8cbd9d3827c09d1b03]) successfully deleted
I20260810 06:53:16.206895 15412 ts_tablet_manager.cc:1905] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:53:16 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:53:16.214607 15412 ts_tablet_manager.cc:1918] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:16.214865 15412 log.cc:1196] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/db88c85d939d4e13a2522110e5092d45
I20260810 06:53:16.215479 15412 ts_tablet_manager.cc:1939] T db88c85d939d4e13a2522110e5092d45 P d62edb8e04b2464383c438609ad4213e: Deleting consensus metadata
I20260810 06:53:16.217236 14484 catalog_manager.cc:4841] TS d62edb8e04b2464383c438609ad4213e (127.22.11.129:33435): tablet db88c85d939d4e13a2522110e5092d45 (table test-table [id=ceaf532047ac4c8cbd9d3827c09d1b03]) successfully deleted
I20260810 06:53:16.219259 15367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.176763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56308 (local address 127.22.11.129:33435)
0810 06:53:16.176954 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:16.176961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:16.196095 (+ 19134us) server_negotiation.cc:407] Connection header received
0810 06:53:16.197469 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:16.197474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:16.197579 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:16.197653 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:16.202816 (+ 5163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.203935 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.205733 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.206324 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.206432 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.211977 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:16.211997 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:16.212014 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:16.212058 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:16.216320 (+ 4262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:16.217230 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:16.217238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:16.217242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:16.217428 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:16.217865 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:16.217868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:16.217870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:16.218211 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:16.218506 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:16.218811 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 06:53:16.218986 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:53:16.219892 14824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:16.319052 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 14610
I20260810 06:53:16.345359 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 14839
Aug 10 06:53:16 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:53:16.359522 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.346101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.22.11.130:35379)
0810 06:53:16.346303 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:16.346309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:16.346330 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:53:16.346486 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:16.346492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:16.346556 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:16.346638 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:16.347946 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.348709 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.349928 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.350338 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.350458 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.354570 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:16.354583 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:16.354599 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:16.354631 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:16.357176 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:16.357805 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:16.357812 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:16.357814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:16.357933 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:16.358213 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:16.358216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:16.358218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:16.358589 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:16.358801 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:16.359037 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 06:53:16.359194 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:53:16.360185 15076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:16.447392 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 14839
I20260810 06:53:16.471897 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 15098
Aug 10 06:53:16 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:53:16.486470 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:16.472652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50686 (local address 127.22.11.131:42281)
0810 06:53:16.472835 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:16.472840 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:16.472859 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:53:16.473032 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:16.473036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:16.473114 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:16.473198 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:16.474584 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.475333 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.476523 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:16.476971 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:16.477077 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:16.481149 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:16.481165 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:16.481182 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:16.481214 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:16.483774 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:16.484393 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:16.484401 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:16.484403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:16.484548 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:16.485020 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:16.485023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:16.485025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:16.485345 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:16.485641 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:16.485934 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:53:16.486163 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:53:16.487080 15321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:16.572916 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 15098
I20260810 06:53:16.597924 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 14454
I20260810 06:53:16.599089 14551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:16.710289 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 14454
I20260810 06:53:16.738865 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43427
--webserver_interface=127.22.11.190
--webserver_port=43861
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:17.064742 15421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:17.065033 15421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:17.065135 15421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:17.072180 15421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:17.072281 15421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:17.072335 15421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:17.072391 15421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:17.072434 15421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:17.081957 15421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:43427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=43861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:17.083118 15421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:17.084306 15421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:17.107733 15428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:17.107726 15436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:17.107736 15430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:17.108690 15421 server_base.cc:1034] running on GCE node
I20260810 06:53:17.109472 15421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:17.110929 15421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:17.112118 15421 hybrid_clock.cc:648] HybridClock initialized: now 1786344797112056 us; error 50 us; skew 500 ppm
I20260810 06:53:17.112509 15421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:17 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344797, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:17.115947 15421 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:53:17.117584 15421 webserver.cc:460] Webserver started at http://127.22.11.190:43861/ using document root <none> and password file <none>
I20260810 06:53:17.118322 15421 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:17.118458 15421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:17.124199 15421 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260810 06:53:17.127055 15444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:17.128315 15421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:53:17.128479 15421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "8826c08e00bf40c79ba1d9c0d465dccd"
format_stamp: "Formatted at 2026-08-10 06:53:13 on dist-test-slave-f7th"
server_key: "5024bbd9a7b65cb133ec51de8623d515"
server_key_iv: "cefcfb9e81a24f0aac42f5e95bba3ada"
server_key_version: "encryptionkey@0"
I20260810 06:53:17.129532 15421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:17.144058 15421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:17.148288 15421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:17.148551 15421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:17.166157 15421 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:43427
I20260810 06:53:17.166178 15523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:43427 every 8 connection(s)
I20260810 06:53:17.167723 15421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:53:17.172158 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 15421
I20260810 06:53:17.172959 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:33435
--local_ip_for_outbound_sockets=127.22.11.129
--tserver_master_addrs=127.22.11.190:43427
--webserver_port=42457
--webserver_interface=127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:53:17.174832 15526 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:53:17.178140 15526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Bootstrap starting.
I20260810 06:53:17.197314 15526 log.cc:824] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:17.209172 15526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:53:17.209653 15526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Bootstrap complete.
I20260810 06:53:17.218838 15526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:17.219645 15526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8826c08e00bf40c79ba1d9c0d465dccd, State: Initialized, Role: FOLLOWER
I20260810 06:53:17.220202 15526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:17.220424 15526 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:53:17.220541 15526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:53:17.220716 15526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:53:17.224339 15526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:17.224865 15526 leader_election.cc:302] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8826c08e00bf40c79ba1d9c0d465dccd; no voters:
I20260810 06:53:17.225468 15526 leader_election.cc:288] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:53:17.225603 15538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:53:17.226864 15538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 LEADER]: Becoming Leader. State: Replica: 8826c08e00bf40c79ba1d9c0d465dccd, State: Running, Role: LEADER
I20260810 06:53:17.227439 15538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:17.227764 15526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:17.230175 15540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8826c08e00bf40c79ba1d9c0d465dccd. Latest consensus state: current_term: 2 leader_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } } }
I20260810 06:53:17.230229 15539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } } }
I20260810 06:53:17.230552 15540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:17.230585 15539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:17.231395 15545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:53:17.258271 15545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:53:17.259673 15545 catalog_manager.cc:1259] Loaded cluster ID: a209bb5335e04b4ba6a3e6962b78fb81
I20260810 06:53:17.259771 15545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:53:17.262914 15545 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 06:53:17.280647 15545 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 06:53:17.289608 15545 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:53:17.291321 15545 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Loaded TSK: 0
I20260810 06:53:17.292901 15545 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:53:17.523341 15529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:17.523612 15529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:17.523744 15529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:17.530953 15529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:17.531077 15529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:17.531232 15529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:53:17.540758 15529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:33435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=42457
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:17.541929 15529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:17.543200 15529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:17.556231 15577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:17.556264 15583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:17.557189 15575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:17.570497 15529 server_base.cc:1034] running on GCE node
I20260810 06:53:17.571245 15529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:17.572368 15529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:17.573616 15529 hybrid_clock.cc:648] HybridClock initialized: now 1786344797573545 us; error 59 us; skew 500 ppm
I20260810 06:53:17.573977 15529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:17 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344797, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:17.577891 15529 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:53:17.579540 15529 webserver.cc:460] Webserver started at http://127.22.11.129:42457/ using document root <none> and password file <none>
I20260810 06:53:17.580495 15529 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:17.580690 15529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:17.587661 15529 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260810 06:53:17.590797 15599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:17.592286 15529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:53:17.592460 15529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "d62edb8e04b2464383c438609ad4213e"
format_stamp: "Formatted at 2026-08-10 06:53:14 on dist-test-slave-f7th"
server_key: "5d67f73707987521e0cfa2c2fa2f15e1"
server_key_iv: "a6fce0bfbc13d6c36e529ce105eb2b8c"
server_key_version: "encryptionkey@0"
I20260810 06:53:17.593763 15529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:17.617300 15529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:17.621178 15529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:17.621405 15529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:17.622288 15529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:17.623555 15529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:17.623642 15529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:17.623724 15529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:17.623778 15529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:17.654403 15529 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:33435
I20260810 06:53:17.654461 15797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:33435 every 8 connection(s)
I20260810 06:53:17.655794 15529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:53:17.661706 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 15529
I20260810 06:53:17.662484 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:35379
--local_ip_for_outbound_sockets=127.22.11.130
--tserver_master_addrs=127.22.11.190:43427
--webserver_port=44865
--webserver_interface=127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:53:17 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344797, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:17.678964 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:17.658507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:47821 (local address 127.22.11.190:43427)
0810 06:53:17.659044 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:17.659056 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:17.659642 (+ 586us) server_negotiation.cc:407] Connection header received
0810 06:53:17.661867 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:17.661885 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:17.662309 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:17.662714 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:17.664135 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:17.665838 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:17.667380 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:17.668267 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:17.668438 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:17.671642 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:17.671651 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:17.671668 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:17.671709 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:17.675032 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:17.675826 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:17.675831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:17.675836 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:17.675991 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:17.676404 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:17.676408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:17.676410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:17.676870 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:17.677010 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:17.677827 (+ 817us) server_negotiation.cc:299] Negotiation successful
0810 06:53:17.678115 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":225,"threads_started":1}
I20260810 06:53:17.679246 15803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:17.658689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:47821)
0810 06:53:17.659478 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:17.659530 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:17.661271 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:17.662902 (+ 1631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:17.662910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:17.663390 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:17.663904 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:17.663921 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:17.666021 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:17.666025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:17.667231 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:17.667244 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:17.668478 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:17.668483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:17.668727 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:17.669326 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:17.669353 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:17.671377 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:17.675179 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:17.675187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:17.675199 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:17.675722 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:17.676088 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:17.676093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:17.676097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:17.676312 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:17.677003 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:17.677011 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:17.677392 (+ 381us) client_negotiation.cc:769] Sending connection context
0810 06:53:17.677663 (+ 271us) client_negotiation.cc:240] Negotiation successful
0810 06:53:17.678111 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":320,"threads_started":1}
I20260810 06:53:17.682230 15799 heartbeater.cc:347] Connected to a master server at 127.22.11.190:43427
I20260810 06:53:17.682498 15799 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:17.683166 15799 heartbeater.cc:511] Master 127.22.11.190:43427 requested a full tablet report, sending...
I20260810 06:53:17.685417 15467 ts_manager.cc:194] Registered new tserver with Master: d62edb8e04b2464383c438609ad4213e (127.22.11.129:33435)
I20260810 06:53:17.687556 15467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:47821
W20260810 06:53:17.991343 15806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:17.991608 15806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:17.991708 15806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:17.998696 15806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:17.998808 15806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:17.998940 15806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:53:18.008178 15806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:35379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=44865
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:18.009367 15806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:18.010619 15806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:18.022960 15813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:18.023042 15814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:18.023073 15817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:18.024127 15806 server_base.cc:1034] running on GCE node
I20260810 06:53:18.024772 15806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:18.025621 15806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:18.026816 15806 hybrid_clock.cc:648] HybridClock initialized: now 1786344798026773 us; error 31 us; skew 500 ppm
I20260810 06:53:18.027124 15806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:18 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344798, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:18.030257 15806 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:53:18.031569 15806 webserver.cc:460] Webserver started at http://127.22.11.130:44865/ using document root <none> and password file <none>
I20260810 06:53:18.032218 15806 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:18.032348 15806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:18.037500 15806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:53:18.039985 15827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:18.041256 15806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:53:18.041405 15806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "71e18bbe13084a968c7384a158972291"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "52f6937dc7a811e62a8d1ce1ec6aa359"
server_key_iv: "a5185953ae1bdae05154460d329290e5"
server_key_version: "encryptionkey@0"
I20260810 06:53:18.042327 15806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:18.063072 15806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:18.066854 15806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:18.067099 15806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:18.068019 15806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:18.069361 15806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:18.069455 15806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:18.069562 15806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:18.069633 15806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:18.099732 15806 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35379
I20260810 06:53:18.099805 16015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35379 every 8 connection(s)
I20260810 06:53:18.101167 15806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:53:18.102046 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 15806
I20260810 06:53:18.102756 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:42281
--local_ip_for_outbound_sockets=127.22.11.131
--tserver_master_addrs=127.22.11.190:43427
--webserver_port=41345
--webserver_interface=127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:53:18 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344798, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:18.125399 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.103848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:39271 (local address 127.22.11.190:43427)
0810 06:53:18.104130 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.104136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.105323 (+ 1187us) server_negotiation.cc:407] Connection header received
0810 06:53:18.107596 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.107600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.107672 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.107760 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:18.109660 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.110339 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.114847 (+ 4508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.115492 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.115650 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.119821 (+ 4171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:18.119825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:18.119828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:18.119862 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:18.122316 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.123194 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.123198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.123200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.123314 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.123729 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.123731 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.123733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.123978 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:18.124093 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.124960 (+ 867us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.125143 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260810 06:53:18.126731 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.104352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:39271)
0810 06:53:18.105151 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:18.105203 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:18.107307 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:18.108022 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:18.108033 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:18.108762 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:18.109493 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.109515 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.110485 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.110490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.114666 (+ 4176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.114679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.115752 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.115760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.116040 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.116736 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:18.116769 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:18.119588 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:18.122446 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:18.122454 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:18.122469 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:18.123095 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:18.123409 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:18.123413 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:18.123416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:18.123640 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:18.124106 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:18.124114 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:18.124556 (+ 442us) client_negotiation.cc:769] Sending connection context
0810 06:53:18.124868 (+ 312us) client_negotiation.cc:240] Negotiation successful
0810 06:53:18.125366 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":329,"threads_started":1}
I20260810 06:53:18.128374 16017 heartbeater.cc:347] Connected to a master server at 127.22.11.190:43427
I20260810 06:53:18.128695 16017 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:18.129483 16017 heartbeater.cc:511] Master 127.22.11.190:43427 requested a full tablet report, sending...
I20260810 06:53:18.131007 15467 ts_manager.cc:194] Registered new tserver with Master: 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379)
I20260810 06:53:18.132138 15467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:39271
W20260810 06:53:18.440958 16021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:18.441200 16021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:18.441301 16021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:18.448374 16021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:18.448468 16021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:18.448586 16021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:53:18.457876 16021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:42281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=41345
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:18.458986 16021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:18.460220 16021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:18.472383 16033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:18.472455 16027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:18.472422 16028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:18.473203 16021 server_base.cc:1034] running on GCE node
I20260810 06:53:18.473731 16021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:18.475153 16021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:18.476333 16021 hybrid_clock.cc:648] HybridClock initialized: now 1786344798476278 us; error 52 us; skew 500 ppm
I20260810 06:53:18.476588 16021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:18 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344798, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:18.479453 16021 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:53:18.480757 16021 webserver.cc:460] Webserver started at http://127.22.11.131:41345/ using document root <none> and password file <none>
I20260810 06:53:18.481355 16021 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:18.481484 16021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:18.486622 16021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 06:53:18.489214 16048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:18.490378 16021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:53:18.490528 16021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "92d85f473aee4d828e604a0d4582309d"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "84bdf50704dcb45f4968ee857ed9fb19"
server_key_iv: "9d3b5df5f4dcd9f6e0cb933b914526a4"
server_key_version: "encryptionkey@0"
I20260810 06:53:18.491438 16021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:18.506112 16021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:18.509683 16021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:18.509918 16021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:18.510820 16021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:18.512123 16021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:18.512216 16021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:18.512316 16021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:18.512388 16021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:18.542276 16021 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:42281
I20260810 06:53:18.542339 16253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:42281 every 8 connection(s)
I20260810 06:53:18.543694 16021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:53:18.545527 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 16021
Aug 10 06:53:18 dist-test-slave-f7th krb5kdc[14430](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344798, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:18.561060 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.546324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.22.11.190:43427)
0810 06:53:18.546820 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.546827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.547291 (+ 464us) server_negotiation.cc:407] Connection header received
0810 06:53:18.547540 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.547546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.547637 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.547753 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:18.549328 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.550451 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.551854 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.552396 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.552486 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.555646 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:18.555652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:18.555657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:18.555690 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:18.558322 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.559098 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.559103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.559105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.559240 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.559648 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.559651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.559653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.559986 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:18.560205 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.560542 (+ 337us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.560737 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":296,"threads_started":1}
I20260810 06:53:18.564090 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.546200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:51307 (local address 127.22.11.190:43427)
0810 06:53:18.546611 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.546617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.547783 (+ 1166us) server_negotiation.cc:407] Connection header received
0810 06:53:18.549799 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.549803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.549864 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.549928 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:18.551479 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.552217 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.554360 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.554762 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.554850 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.558318 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:18.558322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:18.558325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:18.558355 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:18.561322 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.562054 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.562059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.562063 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.562172 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.562574 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.562580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.562583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.562823 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:18.562944 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.563587 (+ 643us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.563780 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260810 06:53:18.564949 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.546809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:51307)
0810 06:53:18.547643 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:18.547700 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:18.549531 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:18.550170 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:18.550180 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:18.550784 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:18.551326 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.551343 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.552357 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.552363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.554225 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.554239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.554975 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.554982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.555261 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.555891 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:18.555912 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:18.558126 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:18.561458 (+ 3332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:18.561467 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:18.561481 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:18.561935 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:18.562272 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:18.562277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:18.562281 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:18.562458 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:18.562944 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:18.562952 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:18.563243 (+ 291us) client_negotiation.cc:769] Sending connection context
0810 06:53:18.563452 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 06:53:18.563905 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":317,"threads_started":1}
I20260810 06:53:18.566079 16257 heartbeater.cc:347] Connected to a master server at 127.22.11.190:43427
I20260810 06:53:18.566349 16257 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:18.567041 16257 heartbeater.cc:511] Master 127.22.11.190:43427 requested a full tablet report, sending...
I20260810 06:53:18.568326 15467 ts_manager.cc:194] Registered new tserver with Master: 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281)
I20260810 06:53:18.569115 15467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:51307
I20260810 06:53:18.575171 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:53:18.581187 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.576032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.22.11.190:43427)
0810 06:53:18.576213 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.576219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.576526 (+ 307us) server_negotiation.cc:407] Connection header received
0810 06:53:18.576716 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.576720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.576833 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.576934 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:53:18.577287 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.577995 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.579341 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.579717 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.579801 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.580180 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:53:18.580737 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:53:18.580843 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.580919 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.580976 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:53:18.581972 15467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:53:18.588896 15467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:53:18.625368 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.599833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35379 (local address 127.0.0.1:53420)
0810 06:53:18.608980 (+ 9147us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:18.609017 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:18.609175 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:18.618765 (+ 9590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:18.618770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:18.618794 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:18.619133 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.619147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.621145 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.621150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.622796 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.622809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.624301 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.624307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.624509 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.624562 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:53:18.624721 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 06:53:18.624933 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8930,"mutex_wait_us":3,"thread_start_us":16198,"threads_started":1}
I20260810 06:53:18.625691 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.600327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.22.11.129:33435)
0810 06:53:18.615558 (+ 15231us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.615567 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.616330 (+ 763us) server_negotiation.cc:407] Connection header received
0810 06:53:18.617180 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.617187 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.617399 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.617603 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:18.618625 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.620505 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.623082 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.624656 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.624781 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.624969 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.625262 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.625419 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15052,"thread_start_us":15232,"threads_started":1}
I20260810 06:53:18.625927 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.602279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50700 (local address 127.22.11.131:42281)
0810 06:53:18.616189 (+ 13910us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.616197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.617296 (+ 1099us) server_negotiation.cc:407] Connection header received
0810 06:53:18.617437 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.617444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.617691 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.617886 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:18.618661 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.620120 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.622771 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.624261 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.624410 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.624654 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.625465 (+ 811us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.625625 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13759,"thread_start_us":13602,"threads_started":1}
I20260810 06:53:18.625984 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.600120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53420 (local address 127.22.11.130:35379)
0810 06:53:18.617816 (+ 17696us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.617823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.617853 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:53:18.617942 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.617948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.618178 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.618777 (+ 599us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:18.619350 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.621028 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.622987 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.624048 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.624261 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.624530 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.625466 (+ 936us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.625625 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17538,"thread_start_us":17449,"threads_started":1}
I20260810 06:53:18.626449 16278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.600006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:33435 (local address 127.0.0.1:56324)
0810 06:53:18.616199 (+ 16193us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:18.616222 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:18.616348 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:18.617865 (+ 1517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:18.617873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:18.617914 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:18.618392 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.618407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.620658 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.620662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.622916 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.622929 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.624885 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.624893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.625084 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.625108 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:53:18.625190 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:53:18.625323 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16061,"mutex_wait_us":25,"spinlock_wait_cycles":1408,"thread_start_us":18403,"threads_started":1}
I20260810 06:53:18.626502 16279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.601713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:42281 (local address 127.0.0.1:50700)
0810 06:53:18.617187 (+ 15474us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:18.617204 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:18.617315 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:18.617876 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:18.617880 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:18.617925 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:18.618438 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.618455 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.620249 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.620254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.622589 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.622604 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.624814 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.624821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.624978 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.625008 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 06:53:18.625107 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 06:53:18.625221 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15361,"mutex_wait_us":3,"spinlock_wait_cycles":2816,"thread_start_us":6921,"threads_started":1}
I20260810 06:53:18.630043 16126 tablet_service.cc:1467] Processing CreateTablet for tablet 57fa734694cb4227aff8b66bef12527e (DEFAULT_TABLE table=test-table [id=c263ce9fd5444821a34ea6152e100d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:18.630124 15901 tablet_service.cc:1467] Processing CreateTablet for tablet 57fa734694cb4227aff8b66bef12527e (DEFAULT_TABLE table=test-table [id=c263ce9fd5444821a34ea6152e100d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:18.630498 15677 tablet_service.cc:1467] Processing CreateTablet for tablet 57fa734694cb4227aff8b66bef12527e (DEFAULT_TABLE table=test-table [id=c263ce9fd5444821a34ea6152e100d13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:18.631356 16126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57fa734694cb4227aff8b66bef12527e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:18.631536 15901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57fa734694cb4227aff8b66bef12527e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:18.631750 15677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57fa734694cb4227aff8b66bef12527e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:18.643651 16286 tablet_bootstrap.cc:492] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Bootstrap starting.
I20260810 06:53:18.646857 16288 tablet_bootstrap.cc:492] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Bootstrap starting.
I20260810 06:53:18.647997 16290 tablet_bootstrap.cc:492] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Bootstrap starting.
I20260810 06:53:18.648577 16286 tablet_bootstrap.cc:654] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:18.650144 16286 log.cc:824] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:18.650701 16288 tablet_bootstrap.cc:654] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:18.650908 16290 tablet_bootstrap.cc:654] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:18.651927 16288 log.cc:824] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:18.652041 16290 log.cc:824] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:18.655452 16288 tablet_bootstrap.cc:492] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: No bootstrap required, opened a new log
I20260810 06:53:18.655756 16288 ts_tablet_manager.cc:1397] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:53:18.653581 16286 tablet_bootstrap.cc:492] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: No bootstrap required, opened a new log
I20260810 06:53:18.657107 16286 ts_tablet_manager.cc:1397] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.002s
I20260810 06:53:18.661665 16290 tablet_bootstrap.cc:492] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: No bootstrap required, opened a new log
I20260810 06:53:18.661978 16290 ts_tablet_manager.cc:1397] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Time spent bootstrapping tablet: real 0.014s user 0.013s sys 0.000s
I20260810 06:53:18.667557 16288 raft_consensus.cc:348] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.668083 16288 raft_consensus.cc:374] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:18.668233 16288 raft_consensus.cc:729] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62edb8e04b2464383c438609ad4213e, State: Initialized, Role: FOLLOWER
I20260810 06:53:18.668689 16286 raft_consensus.cc:348] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.669117 16286 raft_consensus.cc:374] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:18.669003 16288 consensus_queue.cc:260] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.669216 16286 raft_consensus.cc:729] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d85f473aee4d828e604a0d4582309d, State: Initialized, Role: FOLLOWER
I20260810 06:53:18.669849 16286 consensus_queue.cc:260] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.670217 15799 heartbeater.cc:503] Master 127.22.11.190:43427 was elected leader, sending a full tablet report...
I20260810 06:53:18.670884 16288 ts_tablet_manager.cc:1428] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260810 06:53:18.672106 16257 heartbeater.cc:503] Master 127.22.11.190:43427 was elected leader, sending a full tablet report...
I20260810 06:53:18.672037 16290 raft_consensus.cc:348] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.672437 16290 raft_consensus.cc:374] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:18.672538 16290 raft_consensus.cc:729] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71e18bbe13084a968c7384a158972291, State: Initialized, Role: FOLLOWER
I20260810 06:53:18.672786 16286 ts_tablet_manager.cc:1428] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260810 06:53:18.673167 16290 consensus_queue.cc:260] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.674247 16017 heartbeater.cc:503] Master 127.22.11.190:43427 was elected leader, sending a full tablet report...
I20260810 06:53:18.674837 16290 ts_tablet_manager.cc:1428] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260810 06:53:18.795262 16259 tablet.cc:2369] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:53:18.826534 16298 raft_consensus.cc:482] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:53:18.826845 16298 raft_consensus.cc:504] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.828663 16298 leader_election.cc:288] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379), 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281)
I20260810 06:53:18.837417 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.829266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:33935 (local address 127.22.11.130:35379)
0810 06:53:18.829450 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.829456 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.830109 (+ 653us) server_negotiation.cc:407] Connection header received
0810 06:53:18.830321 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.830326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.830403 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.830493 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:18.831229 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.832304 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.835142 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.836350 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.836516 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.836589 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.837041 (+ 452us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.837169 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:53:18.837417 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.829136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:57359 (local address 127.22.11.131:42281)
0810 06:53:18.829336 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.829342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.830193 (+ 851us) server_negotiation.cc:407] Connection header received
0810 06:53:18.830320 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.830325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.830403 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.830493 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:18.831229 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.832305 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.835144 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.836356 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.836509 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.836585 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.837015 (+ 430us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.837157 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:53:18.837653 16301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.829067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:35379 (local address 127.22.11.129:33935)
0810 06:53:18.829988 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:18.830019 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:18.830184 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:18.830530 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:18.830535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:18.830569 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:18.831084 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.831098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.832424 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.832428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.834976 (+ 2548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.835003 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.836602 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.836608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.836827 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.836892 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 06:53:18.836984 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:53:18.837149 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"mutex_wait_us":178,"spinlock_wait_cycles":26240,"thread_start_us":585,"threads_started":1}
I20260810 06:53:18.837674 16302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.829093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:42281 (local address 127.22.11.129:57359)
0810 06:53:18.830099 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:18.830113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:18.830205 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:18.830530 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:18.830535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:18.830577 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:18.831053 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.831067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.832420 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.832426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.835013 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:18.835023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.836575 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.836586 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:18.836838 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.836886 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 06:53:18.836984 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 06:53:18.837149 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"mutex_wait_us":170,"spinlock_wait_cycles":3840,"thread_start_us":424,"threads_started":1}
I20260810 06:53:18.838810 15934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fa734694cb4227aff8b66bef12527e" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e18bbe13084a968c7384a158972291" is_pre_election: true
I20260810 06:53:18.838781 16171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fa734694cb4227aff8b66bef12527e" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d85f473aee4d828e604a0d4582309d" is_pre_election: true
I20260810 06:53:18.839357 15934 raft_consensus.cc:2454] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 0.
I20260810 06:53:18.839357 16171 raft_consensus.cc:2454] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 0.
I20260810 06:53:18.840060 15604 leader_election.cc:302] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71e18bbe13084a968c7384a158972291, d62edb8e04b2464383c438609ad4213e; no voters:
I20260810 06:53:18.840461 16298 raft_consensus.cc:2781] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:53:18.840596 16298 raft_consensus.cc:482] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:53:18.840716 16298 raft_consensus.cc:3037] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:18.842847 16298 raft_consensus.cc:504] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.843384 16298 leader_election.cc:288] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 election: Requested vote from peers 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379), 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281)
I20260810 06:53:18.843789 15934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fa734694cb4227aff8b66bef12527e" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71e18bbe13084a968c7384a158972291"
I20260810 06:53:18.843833 16171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57fa734694cb4227aff8b66bef12527e" candidate_uuid: "d62edb8e04b2464383c438609ad4213e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d85f473aee4d828e604a0d4582309d"
I20260810 06:53:18.843968 15934 raft_consensus.cc:3037] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:18.844002 16171 raft_consensus.cc:3037] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:18.846282 15934 raft_consensus.cc:2454] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 1.
I20260810 06:53:18.846290 16171 raft_consensus.cc:2454] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d62edb8e04b2464383c438609ad4213e in term 1.
I20260810 06:53:18.846628 15604 leader_election.cc:302] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71e18bbe13084a968c7384a158972291, d62edb8e04b2464383c438609ad4213e; no voters:
I20260810 06:53:18.846917 16298 raft_consensus.cc:2781] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:18.847352 16298 raft_consensus.cc:686] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 1 LEADER]: Becoming Leader. State: Replica: d62edb8e04b2464383c438609ad4213e, State: Running, Role: LEADER
I20260810 06:53:18.847781 16298 consensus_queue.cc:237] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } }
I20260810 06:53:18.852195 15465 catalog_manager.cc:5495] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e reported cstate change: term changed from 0 to 1, leader changed from <none> to d62edb8e04b2464383c438609ad4213e (127.22.11.129). New cstate: current_term: 1 leader_uuid: "d62edb8e04b2464383c438609ad4213e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d62edb8e04b2464383c438609ad4213e" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 33435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 } health_report { overall_health: UNKNOWN } } }
W20260810 06:53:18.853032 16019 tablet.cc:2369] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:53:18.908020 15801 tablet.cc:2369] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:53:18.917734 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.912411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56336 (local address 127.22.11.129:33435)
0810 06:53:18.912586 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.912590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.912606 (+ 16us) server_negotiation.cc:407] Connection header received
0810 06:53:18.912730 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.912734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.912809 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.912923 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:53:18.913315 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.914249 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.915615 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.916040 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.916151 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.916606 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:53:18.917146 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:53:18.917280 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.917384 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.917469 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:53:18.925519 16171 raft_consensus.cc:1261] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Refusing update from remote peer d62edb8e04b2464383c438609ad4213e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:18.925525 15934 raft_consensus.cc:1261] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Refusing update from remote peer d62edb8e04b2464383c438609ad4213e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:18.926374 16305 consensus_queue.cc:1035] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d85f473aee4d828e604a0d4582309d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:18.926749 16298 consensus_queue.cc:1035] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [LEADER]: Connected to new peer: Peer: permanent_uuid: "71e18bbe13084a968c7384a158972291" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 35379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:18.937152 16308 mvcc.cc:205] Tried to move back new op lower bound from 7316868296389853184 to 7316868296083988480. Current Snapshot: MvccSnapshot[applied={T|T < 7316868296389853184 or (T in {7316868296389853184})}]
I20260810 06:53:18.941073 16313 mvcc.cc:205] Tried to move back new op lower bound from 7316868296389853184 to 7316868296083988480. Current Snapshot: MvccSnapshot[applied={T|T < 7316868296389853184}]
I20260810 06:53:18.949560 15465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:18.949803 15465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:18.952975 15465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Sending DeleteTablet for 3 replicas of tablet 57fa734694cb4227aff8b66bef12527e
I20260810 06:53:18.954002 15901 tablet_service.cc:1514] Processing DeleteTablet for tablet 57fa734694cb4227aff8b66bef12527e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:18 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:53420
I20260810 06:53:18.954094 16126 tablet_service.cc:1514] Processing DeleteTablet for tablet 57fa734694cb4227aff8b66bef12527e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:18 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50700
I20260810 06:53:18.954687 15677 tablet_service.cc:1514] Processing DeleteTablet for tablet 57fa734694cb4227aff8b66bef12527e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:18 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:56324
I20260810 06:53:18.954783 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 15529
I20260810 06:53:18.955446 16326 tablet_replica.cc:330] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: stopping tablet replica
I20260810 06:53:18.955760 16322 tablet_replica.cc:330] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: stopping tablet replica
I20260810 06:53:18.956207 16322 raft_consensus.cc:2229] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:18.956490 16326 raft_consensus.cc:2229] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:53:18.956830 16322 raft_consensus.cc:2258] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:18.957216 16326 raft_consensus.cc:2258] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:18.957636 16323 tablet_replica.cc:330] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: stopping tablet replica
I20260810 06:53:18.959574 16322 ts_tablet_manager.cc:1905] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:18.960000 16323 raft_consensus.cc:2229] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:18.960489 16323 raft_consensus.cc:2258] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:18.962049 16326 ts_tablet_manager.cc:1905] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:18.962446 16323 ts_tablet_manager.cc:1905] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:18.969247 16322 ts_tablet_manager.cc:1918] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:18.969456 16322 log.cc:1196] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/57fa734694cb4227aff8b66bef12527e
I20260810 06:53:18.970238 16322 ts_tablet_manager.cc:1939] T 57fa734694cb4227aff8b66bef12527e P 71e18bbe13084a968c7384a158972291: Deleting consensus metadata
I20260810 06:53:18.972201 15445 catalog_manager.cc:4841] TS 71e18bbe13084a968c7384a158972291 (127.22.11.130:35379): tablet 57fa734694cb4227aff8b66bef12527e (table test-table [id=c263ce9fd5444821a34ea6152e100d13]) successfully deleted
I20260810 06:53:18.972688 16326 ts_tablet_manager.cc:1918] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:18.972853 16326 log.cc:1196] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/57fa734694cb4227aff8b66bef12527e
I20260810 06:53:18.973388 16326 ts_tablet_manager.cc:1939] T 57fa734694cb4227aff8b66bef12527e P d62edb8e04b2464383c438609ad4213e: Deleting consensus metadata
I20260810 06:53:18.974941 16323 ts_tablet_manager.cc:1918] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:18.974951 15446 catalog_manager.cc:4841] TS d62edb8e04b2464383c438609ad4213e (127.22.11.129:33435): tablet 57fa734694cb4227aff8b66bef12527e (table test-table [id=c263ce9fd5444821a34ea6152e100d13]) successfully deleted
I20260810 06:53:18.975111 16323 log.cc:1196] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/57fa734694cb4227aff8b66bef12527e
I20260810 06:53:18.975533 16323 ts_tablet_manager.cc:1939] T 57fa734694cb4227aff8b66bef12527e P 92d85f473aee4d828e604a0d4582309d: Deleting consensus metadata
I20260810 06:53:18.976825 15447 catalog_manager.cc:4841] TS 92d85f473aee4d828e604a0d4582309d (127.22.11.131:42281): tablet 57fa734694cb4227aff8b66bef12527e (table test-table [id=c263ce9fd5444821a34ea6152e100d13]) successfully deleted
I20260810 06:53:18.978591 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:18.955574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.22.11.129:33435)
0810 06:53:18.955714 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:18.955720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:18.955854 (+ 134us) server_negotiation.cc:407] Connection header received
0810 06:53:18.956045 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:18.956051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:18.956172 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:18.956270 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:18.958053 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.958826 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.961152 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:18.961577 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:18.961683 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:18.964926 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:18.964937 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:18.964948 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:18.964977 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:18.967779 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.968592 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.968600 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.968603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.968793 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:18.977057 (+ 8264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:18.977060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:18.977063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:18.977418 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:18.977750 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:18.978066 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 06:53:18.978293 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260810 06:53:18.979254 15791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:19.074728 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 15529
I20260810 06:53:19.102615 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 15806
I20260810 06:53:19.119424 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:19.103344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.22.11.130:35379)
0810 06:53:19.103524 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:19.103530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:19.103632 (+ 102us) server_negotiation.cc:407] Connection header received
0810 06:53:19.103828 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:19.103834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:19.103916 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:19.104013 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:19.105378 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:19.106311 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:19.107611 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:19.108042 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:19.108153 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:19.113742 (+ 5589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:19.113753 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:19.113770 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:19.113802 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:19.116559 (+ 2757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:19.117394 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:19.117401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:19.117403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:19.117523 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:19.117892 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:19.117895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:19.117897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:19.118265 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:19.118540 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:19.118878 (+ 338us) server_negotiation.cc:299] Negotiation successful
0810 06:53:19.119101 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:53:19.120046 16008 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:19.208350 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 15806
I20260810 06:53:19.236152 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 16021
I20260810 06:53:19.249230 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:19.236915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.22.11.131:42281)
0810 06:53:19.237105 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:19.237110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:19.237131 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:53:19.237259 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:19.237265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:19.237336 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:19.237418 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:19.238841 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:19.239564 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:19.241079 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:19.241461 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:19.241571 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:19.243967 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:19.243982 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:19.244007 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:19.244053 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:19.246509 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:19.247165 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:19.247173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:19.247175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:19.247301 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:19.247655 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:19.247659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:19.247661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:19.248013 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:19.248293 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:19.248580 (+ 287us) server_negotiation.cc:299] Negotiation successful
0810 06:53:19.248913 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:53:19.250058 16246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:19.338266 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 16021
I20260810 06:53:19.364391 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 15421
I20260810 06:53:19.365506 15513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:19.476065 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 15421
I20260810 06:53:19.506572 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43427
--webserver_interface=127.22.11.190
--webserver_port=43861
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:43427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:19.827924 16332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:19.828210 16332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:19.828354 16332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:19.835351 16332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:19.835461 16332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:19.835541 16332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:19.835614 16332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:19.835673 16332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:19.845350 16332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:43427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:43427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=43861
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:19.846609 16332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:19.847851 16332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:19.858949 16341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:19.858862 16344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:19.858903 16342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:19.859750 16332 server_base.cc:1034] running on GCE node
I20260810 06:53:19.860481 16332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:19.861773 16332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:19.862977 16332 hybrid_clock.cc:648] HybridClock initialized: now 1786344799862913 us; error 51 us; skew 500 ppm
I20260810 06:53:19.863282 16332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:19 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344799, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:19.866214 16332 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:53:19.867583 16332 webserver.cc:460] Webserver started at http://127.22.11.190:43861/ using document root <none> and password file <none>
I20260810 06:53:19.868191 16332 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:19.868289 16332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:19.873276 16332 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:53:19.875821 16355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:19.877077 16332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:53:19.877228 16332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "8826c08e00bf40c79ba1d9c0d465dccd"
format_stamp: "Formatted at 2026-08-10 06:53:13 on dist-test-slave-f7th"
server_key: "5024bbd9a7b65cb133ec51de8623d515"
server_key_iv: "cefcfb9e81a24f0aac42f5e95bba3ada"
server_key_version: "encryptionkey@0"
I20260810 06:53:19.878113 16332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:19.902207 16332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:19.905787 16332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:19.906050 16332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:19.923415 16332 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:43427
I20260810 06:53:19.923427 16441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:43427 every 8 connection(s)
I20260810 06:53:19.924885 16332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:53:19.929580 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 16332
I20260810 06:53:19.930562 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:33435
--local_ip_for_outbound_sockets=127.22.11.129
--tserver_master_addrs=127.22.11.190:43427
--webserver_port=42457
--webserver_interface=127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:53:19.932420 16442 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:53:19.936545 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Bootstrap starting.
I20260810 06:53:19.957471 16442 log.cc:824] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:19.973280 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:53:19.973770 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd: Bootstrap complete.
I20260810 06:53:19.982555 16442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:19.983367 16442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8826c08e00bf40c79ba1d9c0d465dccd, State: Initialized, Role: FOLLOWER
I20260810 06:53:19.983933 16442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:19.984134 16442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:53:19.984246 16442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:53:19.984400 16442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:53:19.987979 16442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:19.988435 16442 leader_election.cc:302] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8826c08e00bf40c79ba1d9c0d465dccd; no voters:
I20260810 06:53:19.989037 16442 leader_election.cc:288] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:53:19.989198 16449 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:53:19.990212 16449 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [term 3 LEADER]: Becoming Leader. State: Replica: 8826c08e00bf40c79ba1d9c0d465dccd, State: Running, Role: LEADER
I20260810 06:53:19.990787 16449 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } }
I20260810 06:53:19.991053 16442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:19.993134 16451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8826c08e00bf40c79ba1d9c0d465dccd. Latest consensus state: current_term: 3 leader_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } } }
I20260810 06:53:19.993238 16450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8826c08e00bf40c79ba1d9c0d465dccd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 43427 } } }
I20260810 06:53:19.993543 16451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:19.993579 16450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8826c08e00bf40c79ba1d9c0d465dccd [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:20.000111 16459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:53:20.009250 16459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:53:20.010823 16459 catalog_manager.cc:1259] Loaded cluster ID: a209bb5335e04b4ba6a3e6962b78fb81
I20260810 06:53:20.010962 16459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 06:53:20.013402 16459 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb8b99ce9ad google::LogMessage::Fail() at ??:0
@ 0x7fb8b99d188a google::LogMessage::SendToLog() at ??:0
@ 0x7fb8b99ce4de google::LogMessage::Flush() at ??:0
@ 0x7fb8b99cfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb8cf401512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb8cf3fe976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb8cf486610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb8cf486439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb8bc61a345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb8bc61d91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb8bc61d799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb8bc5f8eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb8bf8d26db start_thread at ??:0
@ 0x7fb8b886861f clone at ??:0
W20260810 06:53:20.309415 16446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:20.309664 16446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:20.309780 16446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:20.316919 16446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:20.317018 16446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:20.317160 16446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:53:20.326552 16446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:33435
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=42457
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:20.327694 16446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:20.329020 16446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:20.341583 16502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:20.341548 16507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:20.341621 16505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:20.342701 16446 server_base.cc:1034] running on GCE node
I20260810 06:53:20.343291 16446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:20.344154 16446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:20.345360 16446 hybrid_clock.cc:648] HybridClock initialized: now 1786344800345303 us; error 36 us; skew 500 ppm
I20260810 06:53:20.345651 16446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:20 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344800, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:20.348672 16446 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:53:20.349996 16446 webserver.cc:460] Webserver started at http://127.22.11.129:42457/ using document root <none> and password file <none>
I20260810 06:53:20.350589 16446 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:20.350690 16446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:20.355557 16446 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260810 06:53:20.358109 16516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:20.359401 16446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:53:20.359556 16446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "d62edb8e04b2464383c438609ad4213e"
format_stamp: "Formatted at 2026-08-10 06:53:14 on dist-test-slave-f7th"
server_key: "5d67f73707987521e0cfa2c2fa2f15e1"
server_key_iv: "a6fce0bfbc13d6c36e529ce105eb2b8c"
server_key_version: "encryptionkey@0"
I20260810 06:53:20.360482 16446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:20.374923 16446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:20.378711 16446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:20.378918 16446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:20.379801 16446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:20.381023 16446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:20.381116 16446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:20.381191 16446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:20.381248 16446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:20.411808 16446 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:33435
I20260810 06:53:20.411862 16714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:33435 every 8 connection(s)
I20260810 06:53:20.413275 16446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:53:20.416666 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 16446
I20260810 06:53:20.417248 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:35379
--local_ip_for_outbound_sockets=127.22.11.130
--tserver_master_addrs=127.22.11.190:43427
--webserver_port=44865
--webserver_interface=127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:53:20.418047 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:20.416061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:39437)
0810 06:53:20.416899 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:20.417175 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":312,"threads_started":1}
W20260810 06:53:20.418674 16715 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:43427 (0 consecutive failures): Network error: Failed to ping master at 127.22.11.190:43427: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
I20260810 06:53:20.419456 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:20.419160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:47069)
0810 06:53:20.419288 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:20.419353 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 06:53:20.420222 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:20.419929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:53199)
0810 06:53:20.420048 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:20.420118 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:53:20.420454 16715 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:53:20.741586 16725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:20.741854 16725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:20.741981 16725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:20.748895 16725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:20.748994 16725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:20.749109 16725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:53:20.758275 16725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:35379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=44865
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:20.759406 16725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:20.760645 16725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:20.772879 16735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:20.773000 16738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:20.772912 16740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:20.773927 16725 server_base.cc:1034] running on GCE node
I20260810 06:53:20.774530 16725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:20.775415 16725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:20.776640 16725 hybrid_clock.cc:648] HybridClock initialized: now 1786344800776571 us; error 33 us; skew 500 ppm
I20260810 06:53:20.776969 16725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:20 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344800, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:20.779891 16725 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:53:20.781329 16725 webserver.cc:460] Webserver started at http://127.22.11.130:44865/ using document root <none> and password file <none>
I20260810 06:53:20.781970 16725 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:20.782089 16725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:20.787171 16725 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:53:20.789764 16752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:20.790972 16725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:53:20.791121 16725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "71e18bbe13084a968c7384a158972291"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "52f6937dc7a811e62a8d1ce1ec6aa359"
server_key_iv: "a5185953ae1bdae05154460d329290e5"
server_key_version: "encryptionkey@0"
I20260810 06:53:20.792052 16725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:20.854427 16725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:20.858500 16725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:20.858750 16725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:20.859736 16725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:20.861099 16725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:20.861205 16725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:20.861310 16725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:20.861384 16725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:20.891194 16725 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:35379
I20260810 06:53:20.891255 16939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:35379 every 8 connection(s)
I20260810 06:53:20.892577 16725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:53:20.893965 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 16725
I20260810 06:53:20.894731 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:42281
--local_ip_for_outbound_sockets=127.22.11.131
--tserver_master_addrs=127.22.11.190:43427
--webserver_port=41345
--webserver_interface=127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:43757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:53:20.897868 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:20.895701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:38361)
0810 06:53:20.896517 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:20.896812 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":289,"threads_started":1}
W20260810 06:53:20.898562 16941 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:43427 (0 consecutive failures): Network error: Failed to ping master at 127.22.11.190:43427: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
I20260810 06:53:20.899369 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:20.899023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:41595)
0810 06:53:20.899174 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:20.899251 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 06:53:20.900255 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:20.899910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:46383)
0810 06:53:20.900044 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:20.900129 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:53:20.900475 16941 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:53:21.219020 16945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:21.219292 16945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:21.219432 16945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:21.226485 16945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:21.226588 16945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:21.226742 16945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:53:21.236140 16945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:43757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:42281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=41345
--tserver_master_addrs=127.22.11.190:43427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:21.237353 16945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:21.238626 16945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:21.252856 16952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:21.253863 16951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:21.254285 16954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:21.254616 16945 server_base.cc:1034] running on GCE node
I20260810 06:53:21.255211 16945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:21.256040 16945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:21.257212 16945 hybrid_clock.cc:648] HybridClock initialized: now 1786344801257161 us; error 46 us; skew 500 ppm
I20260810 06:53:21.257496 16945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:21 dist-test-slave-f7th krb5kdc[14430](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344801, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:21.260365 16945 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:53:21.261744 16945 webserver.cc:460] Webserver started at http://127.22.11.131:41345/ using document root <none> and password file <none>
I20260810 06:53:21.262393 16945 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:21.262524 16945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:21.267562 16945 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 06:53:21.270113 16964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:21.271308 16945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:53:21.271453 16945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "92d85f473aee4d828e604a0d4582309d"
format_stamp: "Formatted at 2026-08-10 06:53:15 on dist-test-slave-f7th"
server_key: "84bdf50704dcb45f4968ee857ed9fb19"
server_key_iv: "9d3b5df5f4dcd9f6e0cb933b914526a4"
server_key_version: "encryptionkey@0"
I20260810 06:53:21.272356 16945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:21.296996 16945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:21.300525 16945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:21.300789 16945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:21.301695 16945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:21.302937 16945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:21.303051 16945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:21.303153 16945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:21.303221 16945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:21.333184 16945 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:42281
I20260810 06:53:21.333243 17147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:42281 every 8 connection(s)
I20260810 06:53:21.334582 16945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:53:21.339244 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:21.337337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:37239)
0810 06:53:21.338123 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:21.338402 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":307,"threads_started":1}
W20260810 06:53:21.339888 17148 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:43427 (0 consecutive failures): Network error: Failed to ping master at 127.22.11.190:43427: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
I20260810 06:53:21.340667 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:21.340314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:49797)
0810 06:53:21.340466 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:21.340537 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:53:21.340731 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 16945
I20260810 06:53:21.341575 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:21.341211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:40761)
0810 06:53:21.341336 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:21.341431 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:53:21.341846 17148 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:53:21.421696 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:21.420934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:39109)
0810 06:53:21.421364 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:21.421471 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":216,"threads_started":1}
I20260810 06:53:21.901684 17158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:21.900930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:53055)
0810 06:53:21.901353 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:21.901460 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":211,"threads_started":1}
I20260810 06:53:22.342985 17159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:22.342255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:34975)
0810 06:53:22.342671 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:22.342764 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":200,"threads_started":1}
I20260810 06:53:22.423112 17160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:22.422434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:41493)
0810 06:53:22.422870 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:22.422948 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":212,"threads_started":1}
I20260810 06:53:22.903172 17162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:22.902482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:55873)
0810 06:53:22.902922 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:22.903005 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":227,"threads_started":1}
I20260810 06:53:23.344389 17163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:23.343723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:43861)
0810 06:53:23.344153 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:23.344233 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":196,"threads_started":1}
I20260810 06:53:23.424502 17165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:23.423835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:52965)
0810 06:53:23.424235 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:23.424312 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":206,"threads_started":1}
I20260810 06:53:23.904644 17166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:23.903939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:34123)
0810 06:53:23.904363 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:23.904440 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":230,"threads_started":1}
I20260810 06:53:24.345803 17167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:24.345140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:52397)
0810 06:53:24.345548 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:24.345628 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":213,"threads_started":1}
I20260810 06:53:24.425961 17168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:24.425317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:54507)
0810 06:53:24.425703 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:24.425776 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":183,"threads_started":1}
I20260810 06:53:24.906080 17169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:24.905395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:43623)
0810 06:53:24.905840 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:24.905917 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":238,"threads_started":1}
I20260810 06:53:25.347272 17171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:25.346549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:35175)
0810 06:53:25.346974 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:25.347078 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":204,"threads_started":1}
I20260810 06:53:25.427465 17172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:25.426762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:44281)
0810 06:53:25.427226 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:25.427306 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":228,"threads_started":1}
I20260810 06:53:25.907793 17175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:25.906962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:34217)
0810 06:53:25.907476 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:25.907604 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":263,"threads_started":1}
I20260810 06:53:26.348889 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:26.348098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:43353)
0810 06:53:26.348596 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:26.348712 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":244,"threads_started":1}
I20260810 06:53:26.429047 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:26.428272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:59139)
0810 06:53:26.428786 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:26.428879 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":249,"threads_started":1}
I20260810 06:53:26.909389 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:26.908651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:50473)
0810 06:53:26.909119 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:26.909208 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":235,"threads_started":1}
I20260810 06:53:27.350457 17180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:27.349752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:35083)
0810 06:53:27.350204 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:27.350293 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":236,"threads_started":1}
I20260810 06:53:27.430732 17181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:27.429952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:41357)
0810 06:53:27.430432 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:27.430516 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":204,"threads_started":1}
I20260810 06:53:27.911027 17182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:27.910210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:42291)
0810 06:53:27.910718 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:27.910809 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":230,"threads_started":1}
I20260810 06:53:28.352003 17183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:28.351285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:41879)
0810 06:53:28.351723 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:28.351802 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":186,"threads_started":1}
I20260810 06:53:28.432222 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:28.431580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:59773)
0810 06:53:28.431983 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:28.432069 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":204,"threads_started":1}
I20260810 06:53:28.912508 17188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:28.911834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:41351)
0810 06:53:28.912233 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:28.912308 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":211,"threads_started":1}
I20260810 06:53:29.353487 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:29.352823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:36949)
0810 06:53:29.353251 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:29.353332 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":229,"threads_started":1}
I20260810 06:53:29.433707 17193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:29.433018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:60585)
0810 06:53:29.433463 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:29.433543 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":206,"threads_started":1}
I20260810 06:53:29.913964 17194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:29.913278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:53655)
0810 06:53:29.913719 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:29.913795 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":219,"threads_started":1}
I20260810 06:53:30.355069 17198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:30.354272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:51381)
0810 06:53:30.354791 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:30.354888 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":219,"threads_started":1}
I20260810 06:53:30.435164 17199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:30.434504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:40173)
0810 06:53:30.434917 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:30.435010 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":214,"threads_started":1}
I20260810 06:53:30.915418 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:30.914762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:49221)
0810 06:53:30.915186 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:30.915264 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":226,"threads_started":1}
I20260810 06:53:31.356594 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:31.355852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:47625)
0810 06:53:31.356336 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:31.356416 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":292,"threads_started":1}
I20260810 06:53:31.436548 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:31.435903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:45453)
0810 06:53:31.436296 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:31.436380 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":193,"threads_started":1}
I20260810 06:53:31.916910 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:31.916197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:35277)
0810 06:53:31.916644 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:31.916732 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":205,"threads_started":1}
I20260810 06:53:32.358067 17210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:32.357410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:39077)
0810 06:53:32.357802 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:32.357893 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":193,"threads_started":1}
I20260810 06:53:32.437981 17211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:32.437333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:53911)
0810 06:53:32.437757 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:32.437831 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":219,"threads_started":1}
I20260810 06:53:32.918467 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:32.917735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:46443)
0810 06:53:32.918236 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:32.918313 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":243,"threads_started":1}
I20260810 06:53:33.359490 17215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:33.358849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:39481)
0810 06:53:33.359259 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:33.359339 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":211,"threads_started":1}
I20260810 06:53:33.439435 17217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:33.438749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:46241)
0810 06:53:33.439175 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:33.439258 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":200,"threads_started":1}
I20260810 06:53:33.920077 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:33.919307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:34837)
0810 06:53:33.919786 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:33.919892 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":247,"threads_started":1}
I20260810 06:53:34.361097 17221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:34.360313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:44163)
0810 06:53:34.360815 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:34.360909 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":243,"threads_started":1}
I20260810 06:53:34.440956 17222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:34.440239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:43435)
0810 06:53:34.440686 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:34.440767 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":211,"threads_started":1}
I20260810 06:53:34.921579 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:34.920867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:48611)
0810 06:53:34.921324 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:34.921401 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":230,"threads_started":1}
I20260810 06:53:35.362654 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:35.361943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:36469)
0810 06:53:35.362388 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:35.362473 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":226,"threads_started":1}
I20260810 06:53:35.442603 17225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:35.441911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:50477)
0810 06:53:35.442334 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:35.442421 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":221,"threads_started":1}
I20260810 06:53:35.923245 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:35.922525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.130:56149)
0810 06:53:35.922973 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:35.923071 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":238,"threads_started":1}
I20260810 06:53:36.349910 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 16446
I20260810 06:53:36.364464 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:36.363683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.131:56613)
0810 06:53:36.364143 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:36.364253 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":195,"threads_started":1}
I20260810 06:53:36.367745 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:36.350971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.22.11.129:33435)
0810 06:53:36.351592 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:36.351611 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:36.351648 (+ 37us) server_negotiation.cc:407] Connection header received
0810 06:53:36.353384 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:36.353395 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:36.353816 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:36.354111 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:36.355497 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:36.356781 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:36.358013 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:36.358641 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:36.358765 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:36.361305 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:36.361313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:36.361326 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:36.361367 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:36.364493 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:36.365277 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:36.365284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:36.365289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:36.365456 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:36.365831 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:36.365835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:36.365837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:36.366365 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:36.366659 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:36.367103 (+ 444us) server_negotiation.cc:299] Negotiation successful
0810 06:53:36.367475 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":255,"threads_started":1}
I20260810 06:53:36.369095 16708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:36.450609 17235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:36.449535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:43427 (local address 127.22.11.129:38557)
0810 06:53:36.450101 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:36.450357 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":262,"threads_started":1}
I20260810 06:53:36.452787 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 16446
I20260810 06:53:36.477202 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 16725
I20260810 06:53:36.494684 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:36.478049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56016 (local address 127.22.11.130:35379)
0810 06:53:36.478589 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:36.478604 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:36.478643 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:53:36.480207 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:36.480217 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:36.480681 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:36.481010 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:36.482505 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:36.483709 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:36.485085 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:36.485743 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:36.485911 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:36.488498 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:36.488506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:36.488519 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:36.488556 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:36.491400 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:36.492099 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:36.492106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:36.492112 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:36.492286 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:36.492744 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:36.492749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:36.492753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:36.493270 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:36.493561 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:36.493999 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 06:53:36.494396 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":231,"threads_started":1}
I20260810 06:53:36.496090 16925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:36.577258 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 16725
I20260810 06:53:36.603648 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 16945
I20260810 06:53:36.621068 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:36.604540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.22.11.131:42281)
0810 06:53:36.605142 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:36.605156 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:36.605198 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:53:36.606783 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:36.606792 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:36.607214 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:36.607597 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:36.609112 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:36.610419 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:36.611640 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:36.612292 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:36.612462 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:36.615019 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:36.615027 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:36.615041 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:36.615080 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:36.617828 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:36.618526 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:36.618533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:36.618539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:36.618702 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:36.619119 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:36.619123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:36.619125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:36.619631 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:36.619929 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:36.620371 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 06:53:36.620773 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":263,"threads_started":1}
I20260810 06:53:36.622421 17140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:36.703251 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 16945
2026-08-10T06:53:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24551 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:53:36 dist-test-slave-f7th krb5kdc[17251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:53:36 dist-test-slave-f7th krb5kdc[17251](info): set up 2 sockets
Aug 10 06:53:36 dist-test-slave-f7th krb5kdc[17251](info): commencing operation
krb5kdc: starting...
W20260810 06:53:37.953552 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:53:38 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:53:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:53:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:53:38.211282 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34089
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:38.537293 17271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:38.537549 17271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:38.537662 17271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:38.544545 17271 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:53:38.544651 17271 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:53:38.544714 17271 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:53:38.544782 17271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:53:38.544852 17271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:38.544945 17271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:38.545015 17271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:53:38.545097 17271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:53:38.554560 17271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:38.555718 17271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:38.556988 17271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:38.568717 17277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:38.568794 17276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:38.568964 17271 server_base.cc:1034] running on GCE node
W20260810 06:53:38.568759 17281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:38.569999 17271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:38.571254 17271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:38.572458 17271 hybrid_clock.cc:648] HybridClock initialized: now 1786344818572391 us; error 37 us; skew 500 ppm
I20260810 06:53:38.572782 17271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:38 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344818, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:38.575654 17271 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:53:38.576934 17271 webserver.cc:460] Webserver started at http://127.22.11.190:42183/ using document root <none> and password file <none>
I20260810 06:53:38.577553 17271 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:38.577637 17271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:38.577879 17271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:38.580592 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "e6ec754c1b704d3eb650c0468629e0ad"
format_stamp: "Formatted at 2026-08-10 06:53:38 on dist-test-slave-f7th"
server_key: "ae74fdfbb5c1d0dea2e2dc15e4a83b94"
server_key_iv: "13a0df2eebecfe6e8a2057aa7800416a"
server_key_version: "encryptionkey@0"
I20260810 06:53:38.581339 17271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "e6ec754c1b704d3eb650c0468629e0ad"
format_stamp: "Formatted at 2026-08-10 06:53:38 on dist-test-slave-f7th"
server_key: "ae74fdfbb5c1d0dea2e2dc15e4a83b94"
server_key_iv: "13a0df2eebecfe6e8a2057aa7800416a"
server_key_version: "encryptionkey@0"
I20260810 06:53:38.586004 17271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:53:38.589090 17289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:38.590265 17271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 06:53:38.590413 17271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "e6ec754c1b704d3eb650c0468629e0ad"
format_stamp: "Formatted at 2026-08-10 06:53:38 on dist-test-slave-f7th"
server_key: "ae74fdfbb5c1d0dea2e2dc15e4a83b94"
server_key_iv: "13a0df2eebecfe6e8a2057aa7800416a"
server_key_version: "encryptionkey@0"
I20260810 06:53:38.590590 17271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:38.608567 17271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:38.612138 17271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:38.612370 17271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:38.629426 17271 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34089
I20260810 06:53:38.629458 17369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34089 every 8 connection(s)
I20260810 06:53:38.630661 17271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:53:38.633411 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 17271
I20260810 06:53:38.633659 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:53:38.634513 22574 external_mini_cluster.cc:1428] Setting key 845ed7d19febfaf488c8f63fce8211be
I20260810 06:53:38.634711 17370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:38.648684 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Bootstrap starting.
Aug 10 06:53:38 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:38.652166 17370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:38.653548 17370 log.cc:824] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:38.656826 17370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: No bootstrap required, opened a new log
I20260810 06:53:38.658596 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:38.636499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.22.11.190:34089)
0810 06:53:38.637290 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:38.637301 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:38.637340 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:53:38.639319 (+ 1979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:38.639340 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:38.639692 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:38.639997 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:38.641841 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:38.643070 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:38.644945 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:38.645646 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:38.645775 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:38.650404 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:38.650415 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:38.650440 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:38.650499 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:38.654158 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:38.654986 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:38.654995 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:38.655002 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:38.655203 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:38.655543 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:38.655548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:38.655551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:38.656244 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:38.656582 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:38.656900 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 06:53:38.657324 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":331,"threads_started":1}
I20260810 06:53:38.665334 17370 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:53:38.665649 17370 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:38.665720 17370 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ec754c1b704d3eb650c0468629e0ad, State: Initialized, Role: FOLLOWER
I20260810 06:53:38.666182 17370 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:53:38.666379 17370 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:53:38.666476 17370 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:53:38.666599 17370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:38.668792 17370 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:53:38.669222 17370 leader_election.cc:302] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6ec754c1b704d3eb650c0468629e0ad; no voters:
I20260810 06:53:38.669703 17370 leader_election.cc:288] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:53:38.669833 17383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:38.670429 17383 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 LEADER]: Becoming Leader. State: Replica: e6ec754c1b704d3eb650c0468629e0ad, State: Running, Role: LEADER
I20260810 06:53:38.670918 17383 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:53:38.671537 17370 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:53:38.673702 17385 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ec754c1b704d3eb650c0468629e0ad. Latest consensus state: current_term: 1 leader_uuid: "e6ec754c1b704d3eb650c0468629e0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } } }
I20260810 06:53:38.674059 17385 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:38.674423 17384 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6ec754c1b704d3eb650c0468629e0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } } }
I20260810 06:53:38.674707 17384 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: This master's current role is: LEADER
I20260810 06:53:38.677507 17402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:53:38.682226 17402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:53:38.690662 17402 catalog_manager.cc:1347] Generated new cluster ID: cf218a0daf8d42e49b832f82da676b0e
I20260810 06:53:38.690757 17402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:53:38.709681 17402 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:53:38.710882 17402 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:53:38.716719 17402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Generated new TSK 0
I20260810 06:53:38.717382 17402 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:53:38.786616 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34089
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:39.119570 17419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:39.119829 17419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:39.119966 17419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:39.127064 17419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:39.127170 17419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:39.127321 17419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:53:39.136580 17419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:39.137818 17419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:39.139092 17419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:39.151531 17429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:39.151611 17428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:39.151801 17431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:39.152534 17419 server_base.cc:1034] running on GCE node
I20260810 06:53:39.153137 17419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:39.154009 17419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:39.155210 17419 hybrid_clock.cc:648] HybridClock initialized: now 1786344819155165 us; error 33 us; skew 500 ppm
I20260810 06:53:39.155499 17419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:39 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:39.158586 17419 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:53:39.159962 17419 webserver.cc:460] Webserver started at http://127.22.11.129:36855/ using document root <none> and password file <none>
I20260810 06:53:39.160758 17419 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:39.160889 17419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:39.161187 17419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:39.164103 17419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "048331157653467277d8aa55b6cdbcd4"
server_key_iv: "53d4863fa0b3c2af359228780df03dba"
server_key_version: "encryptionkey@0"
I20260810 06:53:39.164937 17419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "048331157653467277d8aa55b6cdbcd4"
server_key_iv: "53d4863fa0b3c2af359228780df03dba"
server_key_version: "encryptionkey@0"
I20260810 06:53:39.169880 17419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260810 06:53:39.173223 17441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:39.174527 17419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 06:53:39.174671 17419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "048331157653467277d8aa55b6cdbcd4"
server_key_iv: "53d4863fa0b3c2af359228780df03dba"
server_key_version: "encryptionkey@0"
I20260810 06:53:39.174837 17419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:39.195271 17419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:39.199052 17419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:39.199295 17419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:39.200202 17419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:39.201472 17419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:39.201556 17419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:39.201675 17419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:39.201730 17419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:39.232267 17419 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:41379
I20260810 06:53:39.232316 17637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:41379 every 8 connection(s)
I20260810 06:53:39.233529 17419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:53:39.235167 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 17419
I20260810 06:53:39.235368 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:53:39.236236 22574 external_mini_cluster.cc:1428] Setting key 2ea91b3f5c796c585df2807f9ce796fe
Aug 10 06:53:39 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:39.255061 17646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:39.236750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:55009)
0810 06:53:39.237567 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:39.237611 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:39.239296 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:39.239943 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:39.239950 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:39.240441 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:39.241045 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:39.241068 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.242283 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.242287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:39.243467 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:39.243479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.244299 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.244306 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:39.244537 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:39.245197 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:39.245222 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:39.247629 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:39.250966 (+ 3337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:39.250974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:39.250988 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:39.251518 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:39.251937 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:39.251942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:39.251946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:39.252189 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:39.252904 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:39.252911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:39.253309 (+ 398us) client_negotiation.cc:769] Sending connection context
0810 06:53:39.253599 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 06:53:39.254067 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":322,"threads_started":1}
I20260810 06:53:39.255987 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:39.236076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:55009 (local address 127.22.11.190:34089)
0810 06:53:39.236503 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:39.236507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:39.237721 (+ 1214us) server_negotiation.cc:407] Connection header received
0810 06:53:39.239572 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:39.239577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:39.239656 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:39.239768 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:39.241211 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.242151 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.243599 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.244121 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.244247 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:39.247854 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:39.247860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:39.247865 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:39.247900 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:39.250811 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:39.251658 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:39.251664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:39.251667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:39.251809 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:39.252326 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:39.252331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:39.252334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:39.252695 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:39.252831 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:39.255564 (+ 2733us) server_negotiation.cc:299] Negotiation successful
0810 06:53:39.255749 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":217,"threads_started":1}
I20260810 06:53:39.257205 17640 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34089
I20260810 06:53:39.257529 17640 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:39.258380 17640 heartbeater.cc:511] Master 127.22.11.190:34089 requested a full tablet report, sending...
I20260810 06:53:39.260637 17316 ts_manager.cc:194] Registered new tserver with Master: 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379)
I20260810 06:53:39.262575 17316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:55009
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:53:39.299219 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34089
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:39.620541 17652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:39.620836 17652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:39.620993 17652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:39.628547 17652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:39.628702 17652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:39.628890 17652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:53:39.643709 17652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:39.645304 17652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:39.647061 17652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:39.660400 17660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:39.660436 17658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:39.660399 17657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:39.661603 17652 server_base.cc:1034] running on GCE node
I20260810 06:53:39.662195 17652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:39.663040 17652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:39.664242 17652 hybrid_clock.cc:648] HybridClock initialized: now 1786344819664199 us; error 30 us; skew 500 ppm
I20260810 06:53:39.664533 17652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:39 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:39.667610 17652 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:53:39.668978 17652 webserver.cc:460] Webserver started at http://127.22.11.130:33167/ using document root <none> and password file <none>
I20260810 06:53:39.669628 17652 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:39.669747 17652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:39.670033 17652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:39.673089 17652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "cd01c64ea13446e2812e9fd6dc2186dd"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "8f6fd2e650fcd5c3dc1f0c8957d2f4f8"
server_key_iv: "d41ad306c3a9962ec101f7347fba3607"
server_key_version: "encryptionkey@0"
I20260810 06:53:39.673799 17652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "cd01c64ea13446e2812e9fd6dc2186dd"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "8f6fd2e650fcd5c3dc1f0c8957d2f4f8"
server_key_iv: "d41ad306c3a9962ec101f7347fba3607"
server_key_version: "encryptionkey@0"
I20260810 06:53:39.678817 17652 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:53:39.682157 17671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:39.683398 17652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:53:39.683563 17652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "cd01c64ea13446e2812e9fd6dc2186dd"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "8f6fd2e650fcd5c3dc1f0c8957d2f4f8"
server_key_iv: "d41ad306c3a9962ec101f7347fba3607"
server_key_version: "encryptionkey@0"
I20260810 06:53:39.683753 17652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:39.719821 17652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:39.723881 17652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:39.724149 17652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:39.725111 17652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:39.726387 17652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:39.726475 17652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:39.726606 17652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:39.726661 17652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:39.756201 17652 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:43521
I20260810 06:53:39.756265 17882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:43521 every 8 connection(s)
I20260810 06:53:39.757498 17652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:53:39.762282 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 17652
I20260810 06:53:39.762458 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:53:39.762892 22574 external_mini_cluster.cc:1428] Setting key a545f8cc7ad6ffe9f63526a37df8ded2
Aug 10 06:53:39 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344819, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:39.776221 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:39.759844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:34861 (local address 127.22.11.190:34089)
0810 06:53:39.760269 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:39.760273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:39.761156 (+ 883us) server_negotiation.cc:407] Connection header received
0810 06:53:39.763006 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:39.763011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:39.763092 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:39.763189 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:39.764650 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.765491 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.767049 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.767459 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.767580 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:39.770777 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:39.770782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:39.770786 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:39.770818 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:39.773328 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:39.773987 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:39.773992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:39.773994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:39.774124 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:39.774521 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:39.774525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:39.774527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:39.774808 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:39.774944 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:39.775650 (+ 706us) server_negotiation.cc:299] Negotiation successful
0810 06:53:39.775872 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":229,"threads_started":1}
I20260810 06:53:39.777211 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:39.760284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:34861)
0810 06:53:39.760986 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:39.761044 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:39.762742 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:39.763430 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:39.763438 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:39.763918 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:39.764459 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:39.764476 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.765648 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.765654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:39.766912 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:39.766926 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:39.767670 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:39.767677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:39.767888 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:39.768486 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:39.768508 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:39.770565 (+ 2057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:39.773455 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:39.773464 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:39.773478 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:39.773877 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:39.774228 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:39.774233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:39.774237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:39.774419 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:39.774933 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:39.774941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:39.775255 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 06:53:39.775531 (+ 276us) client_negotiation.cc:240] Negotiation successful
0810 06:53:39.776048 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":309,"threads_started":1}
I20260810 06:53:39.778595 17883 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34089
I20260810 06:53:39.778931 17883 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:39.779801 17883 heartbeater.cc:511] Master 127.22.11.190:34089 requested a full tablet report, sending...
I20260810 06:53:39.781373 17316 ts_manager.cc:194] Registered new tserver with Master: cd01c64ea13446e2812e9fd6dc2186dd (127.22.11.130:43521)
I20260810 06:53:39.782464 17316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:34861
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:53:39.823541 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34089
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:53:40.148886 17894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:53:40.149143 17894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:53:40.149243 17894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:53:40.156343 17894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:53:40.156459 17894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:53:40.156582 17894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:53:40.166333 17894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:53:40.167652 17894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:53:40.168977 17894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:53:40.181833 17902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:40.181715 17903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:53:40.182022 17906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:53:40.182559 17894 server_base.cc:1034] running on GCE node
I20260810 06:53:40.183118 17894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:53:40.183980 17894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:53:40.185173 17894 hybrid_clock.cc:648] HybridClock initialized: now 1786344820185136 us; error 28 us; skew 500 ppm
I20260810 06:53:40.185458 17894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:53:40 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344820, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:53:40.188550 17894 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:53:40.189895 17894 webserver.cc:460] Webserver started at http://127.22.11.131:38047/ using document root <none> and password file <none>
I20260810 06:53:40.190522 17894 fs_manager.cc:359] Metadata directory not provided
I20260810 06:53:40.190646 17894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:53:40.190923 17894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:53:40.193758 17894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "2ca352a055eb44adb32da0787666021d"
format_stamp: "Formatted at 2026-08-10 06:53:40 on dist-test-slave-f7th"
server_key: "c92ea535e85c4d1072ca364cec18c89f"
server_key_iv: "c980915739e7a4700b26b81cc371a94a"
server_key_version: "encryptionkey@0"
I20260810 06:53:40.194530 17894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "2ca352a055eb44adb32da0787666021d"
format_stamp: "Formatted at 2026-08-10 06:53:40 on dist-test-slave-f7th"
server_key: "c92ea535e85c4d1072ca364cec18c89f"
server_key_iv: "c980915739e7a4700b26b81cc371a94a"
server_key_version: "encryptionkey@0"
I20260810 06:53:40.199209 17894 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:53:40.202601 17916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:40.203812 17894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:53:40.203961 17894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "2ca352a055eb44adb32da0787666021d"
format_stamp: "Formatted at 2026-08-10 06:53:40 on dist-test-slave-f7th"
server_key: "c92ea535e85c4d1072ca364cec18c89f"
server_key_iv: "c980915739e7a4700b26b81cc371a94a"
server_key_version: "encryptionkey@0"
I20260810 06:53:40.204128 17894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:53:40.231153 17894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:53:40.238554 17894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:53:40.238833 17894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:53:40.239895 17894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:53:40.241652 17894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:53:40.241747 17894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:40.241869 17894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:53:40.241928 17894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:53:40.266072 17640 heartbeater.cc:503] Master 127.22.11.190:34089 was elected leader, sending a full tablet report...
I20260810 06:53:40.285179 17894 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:42077
I20260810 06:53:40.285276 18100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:42077 every 8 connection(s)
I20260810 06:53:40.286671 17894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:53:40.289654 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 17894
I20260810 06:53:40.289872 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:53:40.290408 22574 external_mini_cluster.cc:1428] Setting key e3048f1fc276673a58e01c66c632e2b5
Aug 10 06:53:40 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344820, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:53:40.310837 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.289823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:34429 (local address 127.22.11.190:34089)
0810 06:53:40.290339 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.290346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.291402 (+ 1056us) server_negotiation.cc:407] Connection header received
0810 06:53:40.293891 (+ 2489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.293896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.293978 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.294074 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:40.295697 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.296982 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.299425 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.300092 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.300220 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.304537 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:40.304543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:40.304548 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:40.304593 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:40.307543 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.308482 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.308487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.308489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.308656 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.309149 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.309152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.309154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.309410 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:40.309525 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.310409 (+ 884us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.310581 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":234,"threads_started":1}
I20260810 06:53:40.312227 18106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.290405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:34429)
0810 06:53:40.291232 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:40.291289 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:40.293485 (+ 2196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:40.294331 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:40.294340 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:40.294941 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:53:40.295515 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.295532 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.297200 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.297206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.299267 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.299291 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.300333 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.300342 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.300670 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.301471 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:53:40.301503 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:53:40.304331 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:53:40.307710 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:40.307720 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:40.307737 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:40.308376 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:40.308765 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:53:40.308770 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:53:40.308774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:53:40.309039 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:53:40.309537 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:53:40.309546 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:53:40.310010 (+ 464us) client_negotiation.cc:769] Sending connection context
0810 06:53:40.310320 (+ 310us) client_negotiation.cc:240] Negotiation successful
0810 06:53:40.310840 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":291,"threads_started":1}
I20260810 06:53:40.313701 18101 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34089
I20260810 06:53:40.314065 18101 heartbeater.cc:464] Registering TS with master...
I20260810 06:53:40.314935 18101 heartbeater.cc:511] Master 127.22.11.190:34089 requested a full tablet report, sending...
I20260810 06:53:40.316530 17316 ts_manager.cc:194] Registered new tserver with Master: 2ca352a055eb44adb32da0787666021d (127.22.11.131:42077)
I20260810 06:53:40.317723 17316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:34429
I20260810 06:53:40.318272 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:53:40.345232 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.330522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42342 (local address 127.22.11.190:34089)
0810 06:53:40.330671 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.330675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.331096 (+ 421us) server_negotiation.cc:407] Connection header received
0810 06:53:40.331246 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.331249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.331307 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.331390 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:40.333189 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.334289 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.335657 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.336323 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.336453 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.339076 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:40.339083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:40.339088 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:40.339131 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:40.342503 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.343132 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.343137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.343142 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.343286 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.343622 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.343627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.343631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.344065 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:40.344319 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.344659 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.344893 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260810 06:53:40.349282 17316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:53:40.355989 17316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:53:40.380506 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.368478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:42077 (local address 127.0.0.1:42372)
0810 06:53:40.369119 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:40.369163 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:40.369372 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:40.370755 (+ 1383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:40.370762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:40.370793 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:40.371174 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.371188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.374084 (+ 2896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.374089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.376448 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.376464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.378409 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.378417 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.378737 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.378798 (+ 61us) client_negotiation.cc:769] Sending connection context
0810 06:53:40.379918 (+ 1120us) client_negotiation.cc:240] Negotiation successful
0810 06:53:40.380122 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":2814,"threads_started":1}
I20260810 06:53:40.381726 18119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.369090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:43521 (local address 127.0.0.1:52168)
0810 06:53:40.369613 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:40.369630 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:40.369741 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:40.371332 (+ 1591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:40.371336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:40.371358 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:40.371785 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.371802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.373880 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.373885 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.378704 (+ 4819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.378720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.380952 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.380959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.381138 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.381173 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:53:40.381259 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:53:40.381378 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":815,"threads_started":1}
I20260810 06:53:40.382495 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.370099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.22.11.130:43521)
0810 06:53:40.370670 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.370677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.370705 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:53:40.370794 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.370800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.371058 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.371306 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:40.372037 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.373705 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.378906 (+ 5201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.380430 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.380561 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.380782 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.381993 (+ 1211us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.382173 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":287,"threads_started":1}
I20260810 06:53:40.382956 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.369479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42372 (local address 127.22.11.131:42077)
0810 06:53:40.370148 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.370155 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.370188 (+ 33us) server_negotiation.cc:407] Connection header received
0810 06:53:40.370277 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.370284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.370523 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.370733 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:40.372236 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.373980 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.376639 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.378183 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.382172 (+ 3989us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.382405 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.382470 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.382631 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":1663,"threads_started":1}
I20260810 06:53:40.387390 17998 tablet_service.cc:1467] Processing CreateTablet for tablet 5be8595ef03f4c7094acade5fb045831 (DEFAULT_TABLE table=test-table [id=1959f66bee85404c874a04be2b1c5cd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:40.387396 17770 tablet_service.cc:1467] Processing CreateTablet for tablet 5be8595ef03f4c7094acade5fb045831 (DEFAULT_TABLE table=test-table [id=1959f66bee85404c874a04be2b1c5cd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:40.388595 18118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.370561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:49798)
0810 06:53:40.380747 (+ 10186us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:40.380762 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:40.380857 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:40.381782 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:40.381787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:40.381811 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:40.381975 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.381985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.384930 (+ 2945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.384936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.386296 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.386315 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.387750 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.387757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.387924 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.387977 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:53:40.388124 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 06:53:40.388258 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10109,"thread_start_us":12430,"threads_started":1}
I20260810 06:53:40.388589 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.371017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.22.11.129:41379)
0810 06:53:40.374569 (+ 3552us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.374577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.380981 (+ 6404us) server_negotiation.cc:407] Connection header received
0810 06:53:40.381091 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.381100 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.381327 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.381500 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:40.383182 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.384731 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.386447 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.387546 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.387673 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.387886 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.388120 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.388279 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3375,"thread_start_us":3222,"threads_started":1}
I20260810 06:53:40.388947 17770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be8595ef03f4c7094acade5fb045831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:40.389245 17998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be8595ef03f4c7094acade5fb045831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:40.391598 17522 tablet_service.cc:1467] Processing CreateTablet for tablet 5be8595ef03f4c7094acade5fb045831 (DEFAULT_TABLE table=test-table [id=1959f66bee85404c874a04be2b1c5cd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:53:40.392696 17522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5be8595ef03f4c7094acade5fb045831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:53:40.409452 18128 tablet_bootstrap.cc:492] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Bootstrap starting.
I20260810 06:53:40.413949 18132 tablet_bootstrap.cc:492] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Bootstrap starting.
I20260810 06:53:40.414078 18130 tablet_bootstrap.cc:492] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Bootstrap starting.
I20260810 06:53:40.417464 18128 tablet_bootstrap.cc:654] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:40.417484 18132 tablet_bootstrap.cc:654] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:40.417491 18130 tablet_bootstrap.cc:654] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Neither blocks nor log segments found. Creating new log.
I20260810 06:53:40.418731 18128 log.cc:824] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:40.418746 18130 log.cc:824] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:40.418741 18132 log.cc:824] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:53:40.423661 18128 tablet_bootstrap.cc:492] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: No bootstrap required, opened a new log
I20260810 06:53:40.423949 18128 ts_tablet_manager.cc:1397] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.004s
I20260810 06:53:40.424103 18132 tablet_bootstrap.cc:492] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: No bootstrap required, opened a new log
I20260810 06:53:40.424373 18132 ts_tablet_manager.cc:1397] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260810 06:53:40.427719 18130 tablet_bootstrap.cc:492] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: No bootstrap required, opened a new log
I20260810 06:53:40.427983 18130 ts_tablet_manager.cc:1397] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260810 06:53:40.434007 18128 raft_consensus.cc:348] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.434430 18128 raft_consensus.cc:374] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:40.434525 18128 raft_consensus.cc:729] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd01c64ea13446e2812e9fd6dc2186dd, State: Initialized, Role: FOLLOWER
I20260810 06:53:40.434544 18132 raft_consensus.cc:348] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.434967 18132 raft_consensus.cc:374] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:40.435070 18132 raft_consensus.cc:729] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25833405a9dc4e06a3d8a2a0b4a58ee2, State: Initialized, Role: FOLLOWER
I20260810 06:53:40.435238 18128 consensus_queue.cc:260] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.435737 18132 consensus_queue.cc:260] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.438390 18130 raft_consensus.cc:348] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.438839 18130 raft_consensus.cc:374] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:53:40.438944 18130 raft_consensus.cc:729] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca352a055eb44adb32da0787666021d, State: Initialized, Role: FOLLOWER
I20260810 06:53:40.440059 17883 heartbeater.cc:503] Master 127.22.11.190:34089 was elected leader, sending a full tablet report...
I20260810 06:53:40.440199 18130 consensus_queue.cc:260] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.440604 18128 ts_tablet_manager.cc:1428] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
I20260810 06:53:40.445474 18132 ts_tablet_manager.cc:1428] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Time spent starting tablet: real 0.021s user 0.015s sys 0.005s
I20260810 06:53:40.447495 18101 heartbeater.cc:503] Master 127.22.11.190:34089 was elected leader, sending a full tablet report...
I20260810 06:53:40.447898 18130 ts_tablet_manager.cc:1428] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Time spent starting tablet: real 0.020s user 0.013s sys 0.007s
W20260810 06:53:40.485709 17643 tablet.cc:2369] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:53:40.509582 17884 tablet.cc:2369] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:53:40.538513 18103 tablet.cc:2369] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:53:40.577617 18139 raft_consensus.cc:482] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:53:40.577908 18139 raft_consensus.cc:504] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.579633 18139 leader_election.cc:288] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ca352a055eb44adb32da0787666021d (127.22.11.131:42077), 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379)
I20260810 06:53:40.586251 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.580007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:46951 (local address 127.22.11.131:42077)
0810 06:53:40.580222 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.580228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.580927 (+ 699us) server_negotiation.cc:407] Connection header received
0810 06:53:40.581098 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.581103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.581179 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.581258 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:40.581873 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.582656 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.584514 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.585303 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.585421 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.585496 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.585884 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.586016 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 06:53:40.586258 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.580142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:41359 (local address 127.22.11.129:41379)
0810 06:53:40.580437 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.580443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.581097 (+ 654us) server_negotiation.cc:407] Connection header received
0810 06:53:40.581264 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.581269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.581346 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.581439 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:53:40.582055 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.582764 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.584515 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.585348 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.585500 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.585578 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.585907 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.586024 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260810 06:53:40.586390 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.579795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:42077 (local address 127.22.11.130:46951)
0810 06:53:40.580800 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:40.580819 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:40.580960 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:40.581300 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:40.581304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:40.581326 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:40.581726 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.581738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.582781 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.582786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.584364 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.584376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.585527 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.585533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.585710 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.585764 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 06:53:40.585836 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 06:53:40.585952 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":88,"thread_start_us":589,"threads_started":1}
I20260810 06:53:40.586745 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.580002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.22.11.130:41359)
0810 06:53:40.580987 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0810 06:53:40.581001 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:53:40.581132 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:53:40.581468 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:53:40.581471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:53:40.581528 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:53:40.581899 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.581911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.582906 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.582909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.584362 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:53:40.584374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.585581 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.585586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:53:40.585761 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.585785 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:53:40.585867 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:53:40.585988 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":23,"spinlock_wait_cycles":45952,"thread_start_us":655,"threads_started":1}
I20260810 06:53:40.587463 18031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be8595ef03f4c7094acade5fb045831" candidate_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca352a055eb44adb32da0787666021d" is_pre_election: true
I20260810 06:53:40.587743 17555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be8595ef03f4c7094acade5fb045831" candidate_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" is_pre_election: true
I20260810 06:53:40.587919 18031 raft_consensus.cc:2454] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd01c64ea13446e2812e9fd6dc2186dd in term 0.
I20260810 06:53:40.588207 17555 raft_consensus.cc:2454] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd01c64ea13446e2812e9fd6dc2186dd in term 0.
I20260810 06:53:40.588533 17676 leader_election.cc:302] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca352a055eb44adb32da0787666021d, cd01c64ea13446e2812e9fd6dc2186dd; no voters:
I20260810 06:53:40.588964 18139 raft_consensus.cc:2781] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:53:40.589114 18139 raft_consensus.cc:482] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:53:40.589216 18139 raft_consensus.cc:3037] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:40.591296 18139 raft_consensus.cc:504] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.591838 18139 leader_election.cc:288] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [CANDIDATE]: Term 1 election: Requested vote from peers 2ca352a055eb44adb32da0787666021d (127.22.11.131:42077), 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379)
I20260810 06:53:40.592173 18031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be8595ef03f4c7094acade5fb045831" candidate_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca352a055eb44adb32da0787666021d"
I20260810 06:53:40.592276 17555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5be8595ef03f4c7094acade5fb045831" candidate_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2"
I20260810 06:53:40.592341 18031 raft_consensus.cc:3037] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:40.592461 17555 raft_consensus.cc:3037] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:53:40.594622 18031 raft_consensus.cc:2454] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd01c64ea13446e2812e9fd6dc2186dd in term 1.
I20260810 06:53:40.594779 17555 raft_consensus.cc:2454] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd01c64ea13446e2812e9fd6dc2186dd in term 1.
I20260810 06:53:40.594986 17676 leader_election.cc:302] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca352a055eb44adb32da0787666021d, cd01c64ea13446e2812e9fd6dc2186dd; no voters:
I20260810 06:53:40.595242 18139 raft_consensus.cc:2781] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:53:40.595693 18139 raft_consensus.cc:686] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 1 LEADER]: Becoming Leader. State: Replica: cd01c64ea13446e2812e9fd6dc2186dd, State: Running, Role: LEADER
I20260810 06:53:40.596144 18139 consensus_queue.cc:237] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:53:40.600734 17315 catalog_manager.cc:5495] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd reported cstate change: term changed from 0 to 1, leader changed from <none> to cd01c64ea13446e2812e9fd6dc2186dd (127.22.11.130). New cstate: current_term: 1 leader_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } health_report { overall_health: UNKNOWN } } }
I20260810 06:53:40.615964 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.611061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52184 (local address 127.22.11.130:43521)
0810 06:53:40.611211 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.611215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.611332 (+ 117us) server_negotiation.cc:407] Connection header received
0810 06:53:40.611516 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.611519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.611594 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.611704 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:53:40.611998 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.612722 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.614052 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.614425 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.614511 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.614969 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:53:40.615455 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:53:40.615578 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.615621 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.615692 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:53:40.623799 18031 raft_consensus.cc:1261] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 1 FOLLOWER]: Refusing update from remote peer cd01c64ea13446e2812e9fd6dc2186dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:40.623799 17555 raft_consensus.cc:1261] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Refusing update from remote peer cd01c64ea13446e2812e9fd6dc2186dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:53:40.624819 18139 consensus_queue.cc:1035] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:40.625102 18149 consensus_queue.cc:1035] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:53:40.638180 18155 mvcc.cc:205] Tried to move back new op lower bound from 7316868385265254400 to 7316868385165488128. Current Snapshot: MvccSnapshot[applied={T|T < 7316868385265254400}]
I20260810 06:53:40.639398 18157 mvcc.cc:205] Tried to move back new op lower bound from 7316868385265254400 to 7316868385165488128. Current Snapshot: MvccSnapshot[applied={T|T < 7316868385265254400}]
I20260810 06:53:40.659384 17316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:40.659703 17316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:53:40.663323 17316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Sending DeleteTablet for 3 replicas of tablet 5be8595ef03f4c7094acade5fb045831
I20260810 06:53:40.664431 17998 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be8595ef03f4c7094acade5fb045831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:40 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:42372
I20260810 06:53:40.664795 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 17419
I20260810 06:53:40.664867 17522 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be8595ef03f4c7094acade5fb045831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:40 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:49798
I20260810 06:53:40.664873 17770 tablet_service.cc:1514] Processing DeleteTablet for tablet 5be8595ef03f4c7094acade5fb045831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:53:40 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52168
I20260810 06:53:40.688990 18173 tablet_replica.cc:330] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: stopping tablet replica
I20260810 06:53:40.689639 18173 raft_consensus.cc:2229] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:40.690263 18173 raft_consensus.cc:2258] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:40.691318 18172 tablet_replica.cc:330] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: stopping tablet replica
I20260810 06:53:40.691737 18172 raft_consensus.cc:2229] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:53:40.692265 18172 raft_consensus.cc:2258] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:40.693050 18167 tablet_replica.cc:330] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: stopping tablet replica
I20260810 06:53:40.693351 18173 ts_tablet_manager.cc:1905] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:40.693621 18167 raft_consensus.cc:2229] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:53:40.694337 18167 raft_consensus.cc:2258] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:53:40.696323 18167 ts_tablet_manager.cc:1905] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:40.698097 18172 ts_tablet_manager.cc:1905] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:53:40.704192 18173 ts_tablet_manager.cc:1918] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:40.704205 18167 ts_tablet_manager.cc:1918] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:40.704373 18173 log.cc:1196] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/5be8595ef03f4c7094acade5fb045831
I20260810 06:53:40.704892 18173 ts_tablet_manager.cc:1939] T 5be8595ef03f4c7094acade5fb045831 P 2ca352a055eb44adb32da0787666021d: Deleting consensus metadata
I20260810 06:53:40.704952 18172 ts_tablet_manager.cc:1918] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:53:40.705113 18172 log.cc:1196] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/5be8595ef03f4c7094acade5fb045831
I20260810 06:53:40.705623 18172 ts_tablet_manager.cc:1939] T 5be8595ef03f4c7094acade5fb045831 P 25833405a9dc4e06a3d8a2a0b4a58ee2: Deleting consensus metadata
I20260810 06:53:40.705724 18167 log.cc:1196] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/5be8595ef03f4c7094acade5fb045831
I20260810 06:53:40.706312 18167 ts_tablet_manager.cc:1939] T 5be8595ef03f4c7094acade5fb045831 P cd01c64ea13446e2812e9fd6dc2186dd: Deleting consensus metadata
I20260810 06:53:40.706779 17291 catalog_manager.cc:4841] TS 2ca352a055eb44adb32da0787666021d (127.22.11.131:42077): tablet 5be8595ef03f4c7094acade5fb045831 (table test-table [id=1959f66bee85404c874a04be2b1c5cd2]) successfully deleted
I20260810 06:53:40.707274 17290 catalog_manager.cc:4841] TS 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379): tablet 5be8595ef03f4c7094acade5fb045831 (table test-table [id=1959f66bee85404c874a04be2b1c5cd2]) successfully deleted
I20260810 06:53:40.708437 17290 catalog_manager.cc:4841] TS cd01c64ea13446e2812e9fd6dc2186dd (127.22.11.130:43521): tablet 5be8595ef03f4c7094acade5fb045831 (table test-table [id=1959f66bee85404c874a04be2b1c5cd2]) successfully deleted
Aug 10 06:53:40 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:53:40.718531 18127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.665947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49150 (local address 127.22.11.129:41379)
0810 06:53:40.666114 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.666120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.691051 (+ 24931us) server_negotiation.cc:407] Connection header received
0810 06:53:40.691742 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.691747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.691833 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.691912 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:40.696805 (+ 4893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.697903 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.704636 (+ 6733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.705304 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.705438 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.712174 (+ 6736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:40.712189 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:40.712207 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:40.712248 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:40.715765 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.716437 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.716446 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.716450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.716639 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.716970 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.716975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.716978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.717433 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:40.717726 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.717941 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.718162 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:53:40.719151 17631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:40.869828 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 17419
I20260810 06:53:40.895869 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 17652
Aug 10 06:53:40 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:53:40.912431 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:40.896716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.22.11.130:43521)
0810 06:53:40.896907 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:40.896913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:40.896934 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:53:40.897004 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:40.897010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:40.897079 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:40.897174 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:40.898552 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.899391 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.900785 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:40.901244 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:40.901346 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:40.906520 (+ 5174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:40.906534 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:40.906549 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:40.906586 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:40.909699 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.910501 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.910508 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.910510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.910642 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:40.911043 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:40.911046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:40.911048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:40.911394 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:40.911640 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:40.911974 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 06:53:40.912148 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:53:40.913151 17874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:41.024863 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 17652
I20260810 06:53:41.051484 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 17894
Aug 10 06:53:41 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:53:41.065795 18121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:53:41.052241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58632 (local address 127.22.11.131:42077)
0810 06:53:41.052401 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:53:41.052405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:53:41.052460 (+ 55us) server_negotiation.cc:407] Connection header received
0810 06:53:41.052664 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:53:41.052668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:53:41.052739 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:53:41.052813 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:53:41.054075 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:41.054780 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:41.056021 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:53:41.056377 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:53:41.056461 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:53:41.060772 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:53:41.060786 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:53:41.060803 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:53:41.060838 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:53:41.063229 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:41.063803 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:41.063810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:41.063812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:41.063944 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:53:41.064320 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:53:41.064323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:53:41.064325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:53:41.064804 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:53:41.065021 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 06:53:41.065298 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 06:53:41.065456 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:53:41.066531 18096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:41.156353 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 17894
I20260810 06:53:41.180011 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 17271
I20260810 06:53:41.181213 17361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:53:41.292635 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 17271
I20260810 06:54:01.319691 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34089
--webserver_interface=127.22.11.190
--webserver_port=42183
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:01.649533 18185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:01.649835 18185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:01.649986 18185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:01.657124 18185 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:54:01.657228 18185 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:54:01.657294 18185 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:54:01.657364 18185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:54:01.657437 18185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:01.657531 18185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:01.657589 18185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:54:01.657662 18185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:54:01.667299 18185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=42183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:01.668540 18185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:01.669890 18185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:01.694409 18197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:01.694490 18196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:01.694593 18185 server_base.cc:1034] running on GCE node
W20260810 06:54:01.694427 18201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:01.695748 18185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:01.697275 18185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:01.698503 18185 hybrid_clock.cc:648] HybridClock initialized: now 1786344841698418 us; error 51 us; skew 500 ppm
I20260810 06:54:01.698873 18185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:01 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344841, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:01.702286 18185 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:54:01.703776 18185 webserver.cc:460] Webserver started at http://127.22.11.190:42183/ using document root <none> and password file <none>
I20260810 06:54:01.704511 18185 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:01.704672 18185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:01.710613 18185 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 06:54:01.713500 18219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:01.714841 18185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:54:01.715004 18185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "e6ec754c1b704d3eb650c0468629e0ad"
format_stamp: "Formatted at 2026-08-10 06:53:38 on dist-test-slave-f7th"
server_key: "ae74fdfbb5c1d0dea2e2dc15e4a83b94"
server_key_iv: "13a0df2eebecfe6e8a2057aa7800416a"
server_key_version: "encryptionkey@0"
I20260810 06:54:01.716049 18185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:01.742748 18185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:01.746654 18185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:01.746927 18185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:01.764465 18185 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34089
I20260810 06:54:01.764488 18306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34089 every 8 connection(s)
I20260810 06:54:01.765906 18185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:54:01.766837 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 18185
I20260810 06:54:01.767720 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:41379
--local_ip_for_outbound_sockets=127.22.11.129
--tserver_master_addrs=127.22.11.190:34089
--webserver_port=36855
--webserver_interface=127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:54:01.773247 18307 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:54:01.776420 18307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Bootstrap starting.
I20260810 06:54:01.795394 18307 log.cc:824] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:01.807327 18307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:54:01.807834 18307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Bootstrap complete.
I20260810 06:54:01.816574 18307 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:01.817400 18307 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ec754c1b704d3eb650c0468629e0ad, State: Initialized, Role: FOLLOWER
I20260810 06:54:01.817988 18307 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:01.818203 18307 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:54:01.818318 18307 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:54:01.818496 18307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 1 FOLLOWER]: Advancing to term 2
I20260810 06:54:01.822427 18307 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:01.822914 18307 leader_election.cc:302] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6ec754c1b704d3eb650c0468629e0ad; no voters:
I20260810 06:54:01.823678 18307 leader_election.cc:288] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 06:54:01.823817 18314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 FOLLOWER]: Leader election won for term 2
I20260810 06:54:01.825088 18314 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 LEADER]: Becoming Leader. State: Replica: e6ec754c1b704d3eb650c0468629e0ad, State: Running, Role: LEADER
I20260810 06:54:01.825695 18314 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:01.826072 18307 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:54:01.829401 18319 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ec754c1b704d3eb650c0468629e0ad. Latest consensus state: current_term: 2 leader_uuid: "e6ec754c1b704d3eb650c0468629e0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } } }
I20260810 06:54:01.829766 18319 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:01.830281 18316 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e6ec754c1b704d3eb650c0468629e0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } } }
I20260810 06:54:01.830601 18316 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:01.838008 18327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:54:01.846323 18327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:54:01.847909 18327 catalog_manager.cc:1259] Loaded cluster ID: cf218a0daf8d42e49b832f82da676b0e
I20260810 06:54:01.848074 18327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:54:01.852305 18327 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:54:01.854036 18327 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Loaded TSK: 0
I20260810 06:54:01.881862 18341 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:54:01.892838 18327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Generated new TSK 1
I20260810 06:54:01.893195 18327 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 06:54:02.143826 18309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:02.144101 18309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:02.144222 18309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:02.151366 18309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:02.151469 18309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:02.151607 18309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:54:02.161119 18309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:41379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=36855
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:02.162328 18309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:02.163640 18309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:02.177143 18351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:02.177192 18352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:02.178200 18309 server_base.cc:1034] running on GCE node
W20260810 06:54:02.178279 18355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:02.178937 18309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:02.179817 18309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:02.181012 18309 hybrid_clock.cc:648] HybridClock initialized: now 1786344842180956 us; error 49 us; skew 500 ppm
I20260810 06:54:02.181308 18309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:02 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344842, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:02.184278 18309 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:54:02.185725 18309 webserver.cc:460] Webserver started at http://127.22.11.129:36855/ using document root <none> and password file <none>
I20260810 06:54:02.186339 18309 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:02.186461 18309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:02.191506 18309 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:54:02.194166 18366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:02.195549 18309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:54:02.195691 18309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "048331157653467277d8aa55b6cdbcd4"
server_key_iv: "53d4863fa0b3c2af359228780df03dba"
server_key_version: "encryptionkey@0"
I20260810 06:54:02.196594 18309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:02.217235 18309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:02.221424 18309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:02.221674 18309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:02.222666 18309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:02.223973 18309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:02.224077 18309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:02.224192 18309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:02.224265 18309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:02.254680 18309 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:41379
I20260810 06:54:02.254745 18548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:41379 every 8 connection(s)
I20260810 06:54:02.256194 18309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:54:02.266098 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 18309
I20260810 06:54:02.266921 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:43521
--local_ip_for_outbound_sockets=127.22.11.130
--tserver_master_addrs=127.22.11.190:34089
--webserver_port=33167
--webserver_interface=127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:54:02 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344842, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:02.279332 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:02.259206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:58623)
0810 06:54:02.260016 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:02.260060 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:02.261803 (+ 1743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:02.263077 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:02.263085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:02.263586 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:02.264119 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:02.264136 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.265821 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.265826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:02.267685 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:02.267700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.268781 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.268787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:02.269060 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:02.269707 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:02.269739 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:02.272272 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:02.275452 (+ 3180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:02.275460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:02.275472 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:02.275998 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:02.276356 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:02.276360 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:02.276364 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:02.276583 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:02.277231 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:02.277238 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:02.277586 (+ 348us) client_negotiation.cc:769] Sending connection context
0810 06:54:02.277867 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 06:54:02.278322 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":348,"threads_started":1}
I20260810 06:54:02.279376 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:02.259012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:58623 (local address 127.22.11.190:34089)
0810 06:54:02.259654 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:02.259672 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:02.260189 (+ 517us) server_negotiation.cc:407] Connection header received
0810 06:54:02.262303 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:02.262316 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:02.262631 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:02.262937 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:02.264307 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.265652 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.267854 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.268521 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.268728 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:02.272533 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:02.272541 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:02.272554 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:02.272589 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:02.275320 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:02.276103 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:02.276109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:02.276114 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:02.276259 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:02.276686 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:02.276689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:02.276691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:02.277104 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:02.277251 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:02.278004 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 06:54:02.278314 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":242,"threads_started":1}
I20260810 06:54:02.282912 18550 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34089
I20260810 06:54:02.283213 18550 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:02.284047 18550 heartbeater.cc:511] Master 127.22.11.190:34089 requested a full tablet report, sending...
I20260810 06:54:02.286414 18252 ts_manager.cc:194] Registered new tserver with Master: 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379)
I20260810 06:54:02.288416 18252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:58623
W20260810 06:54:02.596642 18557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:02.596913 18557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:02.597014 18557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:02.604100 18557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:02.604197 18557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:02.604311 18557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:54:02.613629 18557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:43521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=33167
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:02.614741 18557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:02.615971 18557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:02.628635 18566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:02.628729 18572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:02.628757 18568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:02.629702 18557 server_base.cc:1034] running on GCE node
I20260810 06:54:02.630339 18557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:02.631235 18557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:02.632427 18557 hybrid_clock.cc:648] HybridClock initialized: now 1786344842632358 us; error 60 us; skew 500 ppm
I20260810 06:54:02.632766 18557 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:02 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344842, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:02.635721 18557 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:54:02.637178 18557 webserver.cc:460] Webserver started at http://127.22.11.130:33167/ using document root <none> and password file <none>
I20260810 06:54:02.637806 18557 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:02.637945 18557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:02.643045 18557 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:54:02.645665 18586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:02.646983 18557 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:54:02.647137 18557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "cd01c64ea13446e2812e9fd6dc2186dd"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "8f6fd2e650fcd5c3dc1f0c8957d2f4f8"
server_key_iv: "d41ad306c3a9962ec101f7347fba3607"
server_key_version: "encryptionkey@0"
I20260810 06:54:02.648070 18557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:02.659905 18557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:02.663668 18557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:02.663929 18557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:02.664889 18557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:02.666191 18557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:02.666287 18557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:02.666401 18557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:02.666473 18557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:02.696642 18557 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:43521
I20260810 06:54:02.696708 18778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:43521 every 8 connection(s)
I20260810 06:54:02.698060 18557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:54:02.707741 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 18557
I20260810 06:54:02.708420 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:42077
--local_ip_for_outbound_sockets=127.22.11.131
--tserver_master_addrs=127.22.11.190:34089
--webserver_port=38047
--webserver_interface=127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 06:54:02 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344842, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:02.717911 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:02.700603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:34047 (local address 127.22.11.190:34089)
0810 06:54:02.700795 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:02.700800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:02.702024 (+ 1224us) server_negotiation.cc:407] Connection header received
0810 06:54:02.703818 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:02.703824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:02.703902 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:02.704021 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:02.705550 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.706247 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.708006 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.708439 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.708569 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:02.712461 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:02.712467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:02.712470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:02.712503 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:02.714997 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:02.715782 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:02.715786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:02.715788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:02.715931 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:02.716354 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:02.716357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:02.716359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:02.716608 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:02.716750 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:02.717486 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 06:54:02.717654 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:54:02.719138 18784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:02.701163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:34047)
0810 06:54:02.701863 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:02.701907 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:02.703559 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:02.704240 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:02.704248 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:02.704815 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:02.705383 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:02.705401 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.706400 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.706404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:02.707853 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:02.707876 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:02.708696 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:02.708703 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:02.708955 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:02.709647 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:02.709674 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:02.712227 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:02.715131 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:02.715139 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:02.715153 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:02.715685 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:02.716022 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:02.716026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:02.716029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:02.716263 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:02.716760 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:02.716767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:02.717147 (+ 380us) client_negotiation.cc:769] Sending connection context
0810 06:54:02.717413 (+ 266us) client_negotiation.cc:240] Negotiation successful
0810 06:54:02.717899 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":285,"threads_started":1}
I20260810 06:54:02.720602 18780 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34089
I20260810 06:54:02.720983 18780 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:02.721673 18780 heartbeater.cc:511] Master 127.22.11.190:34089 requested a full tablet report, sending...
I20260810 06:54:02.723124 18252 ts_manager.cc:194] Registered new tserver with Master: cd01c64ea13446e2812e9fd6dc2186dd (127.22.11.130:43521)
I20260810 06:54:02.724141 18252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:34047
W20260810 06:54:03.037850 18785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:03.038121 18785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:03.038223 18785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:03.045259 18785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:03.045357 18785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:03.045472 18785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:54:03.054785 18785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:42077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=38047
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:03.055915 18785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:03.057164 18785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:03.069976 18793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:03.069947 18794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:03.069911 18796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:03.073961 18785 server_base.cc:1034] running on GCE node
I20260810 06:54:03.074584 18785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:03.075476 18785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:03.076691 18785 hybrid_clock.cc:648] HybridClock initialized: now 1786344843076644 us; error 40 us; skew 500 ppm
I20260810 06:54:03.076994 18785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:03 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344843, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:03.079934 18785 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:54:03.081441 18785 webserver.cc:460] Webserver started at http://127.22.11.131:38047/ using document root <none> and password file <none>
I20260810 06:54:03.082126 18785 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:03.082258 18785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:03.087507 18785 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:54:03.090145 18805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:03.091454 18785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:54:03.091603 18785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "2ca352a055eb44adb32da0787666021d"
format_stamp: "Formatted at 2026-08-10 06:53:40 on dist-test-slave-f7th"
server_key: "c92ea535e85c4d1072ca364cec18c89f"
server_key_iv: "c980915739e7a4700b26b81cc371a94a"
server_key_version: "encryptionkey@0"
I20260810 06:54:03.092530 18785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:03.113926 18785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:03.118151 18785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:03.118405 18785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:03.119374 18785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:03.120805 18785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:03.120913 18785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:03.121039 18785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:03.121116 18785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:03.151726 18785 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:42077
I20260810 06:54:03.151789 19009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:42077 every 8 connection(s)
I20260810 06:54:03.153246 18785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:54:03.158492 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 18785
Aug 10 06:54:03 dist-test-slave-f7th krb5kdc[17251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344843, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:03.174150 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.159469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49670 (local address 127.22.11.190:34089)
0810 06:54:03.159969 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.159974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.160369 (+ 395us) server_negotiation.cc:407] Connection header received
0810 06:54:03.160541 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.160544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.160594 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.160715 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:03.162712 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.163413 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.165097 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.165483 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.165588 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.168442 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:03.168450 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:03.168453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:03.168499 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:03.171361 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.172090 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.172093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.172095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.172233 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.172683 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.172688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.172691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.172961 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:03.173202 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.173528 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.173734 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":263,"threads_started":1}
I20260810 06:54:03.174249 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.155655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:47189 (local address 127.22.11.190:34089)
0810 06:54:03.155874 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.155880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.156996 (+ 1116us) server_negotiation.cc:407] Connection header received
0810 06:54:03.159052 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.159057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.159132 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.159239 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:03.161249 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.162342 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.163834 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.164421 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.164545 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.168119 (+ 3574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:03.168124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:03.168128 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:03.168164 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:03.171181 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.171960 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.171966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.171969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.172128 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.172578 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.172582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.172584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.172876 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:03.173014 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.173664 (+ 650us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.173863 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:54:03.175349 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.156137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:47189)
0810 06:54:03.156843 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.156881 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:03.158763 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:03.159516 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:03.159529 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:03.160271 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:03.161090 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.161110 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.162473 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.162477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.163712 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.163723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.164641 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.164649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.164855 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.165442 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:03.165469 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:03.167907 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:03.171327 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:03.171336 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:03.171351 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:03.171808 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:03.172235 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:03.172240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:03.172243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:03.172453 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:03.173000 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:03.173008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:03.173321 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 06:54:03.173560 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 06:54:03.174232 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":263,"threads_started":1}
I20260810 06:54:03.176630 19011 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34089
I20260810 06:54:03.176893 19011 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:03.177546 19011 heartbeater.cc:511] Master 127.22.11.190:34089 requested a full tablet report, sending...
I20260810 06:54:03.178821 18252 ts_manager.cc:194] Registered new tserver with Master: 2ca352a055eb44adb32da0787666021d (127.22.11.131:42077)
I20260810 06:54:03.179574 18252 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:47189
I20260810 06:54:03.188747 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:54:03.194742 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.189588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49672 (local address 127.22.11.190:34089)
0810 06:54:03.189721 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.189725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.190171 (+ 446us) server_negotiation.cc:407] Connection header received
0810 06:54:03.190347 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.190350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.190472 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.190592 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:54:03.190935 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.191635 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.193066 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.193445 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.193534 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.194006 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:54:03.194281 (+ 275us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 06:54:03.194551 (+ 270us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49672: token expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 06:54:03.194845 19016 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49672: token expired
W20260810 06:54:03.195372 18108 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.22.11.190:34089)
I20260810 06:54:03.206923 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.195825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.22.11.190:34089)
0810 06:54:03.195965 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.195969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.196010 (+ 41us) server_negotiation.cc:407] Connection header received
0810 06:54:03.196130 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.196133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.196191 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.196288 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:03.197618 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.198278 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.199547 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.199938 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.200030 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.202419 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:03.202424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:03.202427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:03.202454 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:03.204723 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.205346 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.205350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.205352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.205474 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.205803 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.205807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.205809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.206121 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:03.206332 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.206517 (+ 185us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.206677 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:54:03.210520 18252 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:54:03.217773 18252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:54:03.246372 19032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.229256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:43521 (local address 127.0.0.1:55744)
0810 06:54:03.236211 (+ 6955us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.236259 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:03.236438 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:03.237926 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:03.237932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:03.237960 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:03.238395 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.238409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.240514 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.240520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.243189 (+ 2669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.243206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.245319 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.245325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.245509 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.245559 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 06:54:03.245690 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 06:54:03.245921 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6667,"mutex_wait_us":32,"spinlock_wait_cycles":30976,"thread_start_us":8426,"threads_started":1}
I20260810 06:54:03.246889 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.229339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.22.11.130:43521)
0810 06:54:03.237056 (+ 7717us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.237064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.237092 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:54:03.237192 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.237200 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.237458 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.237671 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:03.238610 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.240343 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.243416 (+ 3073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.245085 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.245236 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.245446 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.245734 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.245978 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7568,"thread_start_us":7440,"threads_started":1}
I20260810 06:54:03.246976 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.230036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57652 (local address 127.22.11.131:42077)
0810 06:54:03.237639 (+ 7603us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.237647 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.238209 (+ 562us) server_negotiation.cc:407] Connection header received
0810 06:54:03.238315 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.238323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.238544 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.238717 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:03.239634 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.241345 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.244055 (+ 2710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.245363 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.245481 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.245707 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.246469 (+ 762us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.246625 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7435,"thread_start_us":7277,"threads_started":1}
I20260810 06:54:03.247440 19030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.229373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:42077 (local address 127.0.0.1:57652)
0810 06:54:03.237927 (+ 8554us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.237943 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:03.238076 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:03.239021 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:03.239026 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:03.239053 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:03.239409 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.239419 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.241520 (+ 2101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.241529 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.243847 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.243863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.245839 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.245844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.246018 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.246055 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 06:54:03.246137 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:54:03.246256 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8442,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":6354,"threads_started":1}
I20260810 06:54:03.251179 18660 tablet_service.cc:1467] Processing CreateTablet for tablet 89dfc23a50754b83b866a98191a42b4c (DEFAULT_TABLE table=test-table [id=e29914bd4faa478e9bb49560802fb412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:03.252897 18660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dfc23a50754b83b866a98191a42b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:03.254277 18906 tablet_service.cc:1467] Processing CreateTablet for tablet 89dfc23a50754b83b866a98191a42b4c (DEFAULT_TABLE table=test-table [id=e29914bd4faa478e9bb49560802fb412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:03.256136 18906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dfc23a50754b83b866a98191a42b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:03.266459 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.238047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55710)
0810 06:54:03.244568 (+ 6521us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.244584 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:03.244909 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:03.258434 (+ 13525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:03.258439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:03.258468 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:03.258776 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.258787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.261803 (+ 3016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.261809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.263808 (+ 1999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.263825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.265713 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.265720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.265892 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.265930 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 06:54:03.265993 (+ 63us) client_negotiation.cc:240] Negotiation successful
0810 06:54:03.266123 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6402,"thread_start_us":6662,"threads_started":1}
I20260810 06:54:03.267742 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.238313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55710 (local address 127.22.11.129:41379)
0810 06:54:03.257862 (+ 19549us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.257870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.257896 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:54:03.257992 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.258000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.258223 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.258420 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:03.258962 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.261654 (+ 2692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.264008 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.265477 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.266974 (+ 1497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.267200 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.267277 (+ 77us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.267428 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19371,"thread_start_us":20306,"threads_started":1}
I20260810 06:54:03.269050 19041 tablet_bootstrap.cc:492] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Bootstrap starting.
I20260810 06:54:03.271492 18445 tablet_service.cc:1467] Processing CreateTablet for tablet 89dfc23a50754b83b866a98191a42b4c (DEFAULT_TABLE table=test-table [id=e29914bd4faa478e9bb49560802fb412]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:03.272037 19041 tablet_bootstrap.cc:654] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:03.272989 18445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89dfc23a50754b83b866a98191a42b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:03.273098 19042 tablet_bootstrap.cc:492] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Bootstrap starting.
I20260810 06:54:03.274132 19041 log.cc:824] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:03.276371 19042 tablet_bootstrap.cc:654] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:03.277415 19041 tablet_bootstrap.cc:492] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: No bootstrap required, opened a new log
I20260810 06:54:03.277684 19041 ts_tablet_manager.cc:1397] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260810 06:54:03.277649 19042 log.cc:824] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:03.280504 19042 tablet_bootstrap.cc:492] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: No bootstrap required, opened a new log
I20260810 06:54:03.280781 19042 ts_tablet_manager.cc:1397] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260810 06:54:03.286425 19046 tablet_bootstrap.cc:492] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Bootstrap starting.
I20260810 06:54:03.287710 19041 raft_consensus.cc:348] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.288148 19041 raft_consensus.cc:374] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:03.288254 19041 raft_consensus.cc:729] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca352a055eb44adb32da0787666021d, State: Initialized, Role: FOLLOWER
I20260810 06:54:03.288867 19041 consensus_queue.cc:260] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.289953 19011 heartbeater.cc:503] Master 127.22.11.190:34089 was elected leader, sending a full tablet report...
I20260810 06:54:03.290540 19041 ts_tablet_manager.cc:1428] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 06:54:03.291324 19046 tablet_bootstrap.cc:654] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:03.291489 18550 heartbeater.cc:503] Master 127.22.11.190:34089 was elected leader, sending a full tablet report...
I20260810 06:54:03.292121 19042 raft_consensus.cc:348] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.292517 19042 raft_consensus.cc:374] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:03.292604 19042 raft_consensus.cc:729] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd01c64ea13446e2812e9fd6dc2186dd, State: Initialized, Role: FOLLOWER
I20260810 06:54:03.292717 19046 log.cc:824] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:03.293323 19042 consensus_queue.cc:260] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.300243 18780 heartbeater.cc:503] Master 127.22.11.190:34089 was elected leader, sending a full tablet report...
I20260810 06:54:03.300277 19047 raft_consensus.cc:482] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:54:03.300509 19047 raft_consensus.cc:504] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.301653 19042 ts_tablet_manager.cc:1428] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Time spent starting tablet: real 0.021s user 0.014s sys 0.005s
I20260810 06:54:03.302371 19046 tablet_bootstrap.cc:492] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: No bootstrap required, opened a new log
I20260810 06:54:03.302356 19047 leader_election.cc:288] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd01c64ea13446e2812e9fd6dc2186dd (127.22.11.130:43521), 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379)
I20260810 06:54:03.302672 19046 ts_tablet_manager.cc:1397] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260810 06:54:03.310040 19014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.302671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:43521 (local address 127.22.11.131:56875)
0810 06:54:03.303422 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.303441 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:03.303555 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:03.304018 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:03.304024 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:03.304052 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:03.304244 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.304256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.305761 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.305767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.307769 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.307785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.309351 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.309358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.309607 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.309642 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:54:03.309722 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:54:03.309836 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":12}
I20260810 06:54:03.310609 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.302991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:56875 (local address 127.22.11.130:43521)
0810 06:54:03.303182 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.303188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.303518 (+ 330us) server_negotiation.cc:407] Connection header received
0810 06:54:03.303684 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.303689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.303761 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.303852 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:03.304416 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.305629 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.307936 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.309191 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.309342 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.309416 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.310346 (+ 930us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.310443 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:54:03.311463 18694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dfc23a50754b83b866a98191a42b4c" candidate_uuid: "2ca352a055eb44adb32da0787666021d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" is_pre_election: true
I20260810 06:54:03.312012 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.303240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:56643 (local address 127.22.11.129:41379)
0810 06:54:03.303460 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.303466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.305630 (+ 2164us) server_negotiation.cc:407] Connection header received
0810 06:54:03.305868 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.305874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.305942 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.306028 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:03.306787 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.307915 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.309753 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.311064 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.311217 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.311286 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.311638 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.311772 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:54:03.311970 18694 raft_consensus.cc:2454] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ca352a055eb44adb32da0787666021d in term 0.
I20260810 06:54:03.312512 19051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.303065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.22.11.131:56643)
0810 06:54:03.305545 (+ 2480us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.305559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:03.305740 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:03.306057 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:03.306062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:03.306088 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:03.306575 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.306589 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.308065 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.308069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.309593 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:03.309607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.311245 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.311252 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:03.311420 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.311454 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:54:03.311613 (+ 159us) client_negotiation.cc:240] Negotiation successful
0810 06:54:03.311823 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2360,"mutex_wait_us":53,"thread_start_us":2204,"threads_started":1}
I20260810 06:54:03.312597 18807 leader_election.cc:302] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca352a055eb44adb32da0787666021d, cd01c64ea13446e2812e9fd6dc2186dd; no voters:
I20260810 06:54:03.312952 19047 raft_consensus.cc:2781] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:54:03.313125 19047 raft_consensus.cc:482] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:54:03.312907 19046 raft_consensus.cc:348] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.313225 19047 raft_consensus.cc:3037] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:03.313323 19046 raft_consensus.cc:374] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:03.313434 19046 raft_consensus.cc:729] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25833405a9dc4e06a3d8a2a0b4a58ee2, State: Initialized, Role: FOLLOWER
I20260810 06:54:03.313776 18483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dfc23a50754b83b866a98191a42b4c" candidate_uuid: "2ca352a055eb44adb32da0787666021d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" is_pre_election: true
I20260810 06:54:03.314082 19046 consensus_queue.cc:260] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.314452 18483 raft_consensus.cc:2454] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ca352a055eb44adb32da0787666021d in term 0.
I20260810 06:54:03.315943 19046 ts_tablet_manager.cc:1428] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260810 06:54:03.315915 19047 raft_consensus.cc:504] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.316514 19047 leader_election.cc:288] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [CANDIDATE]: Term 1 election: Requested vote from peers cd01c64ea13446e2812e9fd6dc2186dd (127.22.11.130:43521), 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379)
I20260810 06:54:03.316908 18694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dfc23a50754b83b866a98191a42b4c" candidate_uuid: "2ca352a055eb44adb32da0787666021d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd01c64ea13446e2812e9fd6dc2186dd"
I20260810 06:54:03.317116 18694 raft_consensus.cc:3037] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:03.317247 18483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89dfc23a50754b83b866a98191a42b4c" candidate_uuid: "2ca352a055eb44adb32da0787666021d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2"
I20260810 06:54:03.317430 18483 raft_consensus.cc:3037] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:03.319713 18483 raft_consensus.cc:2454] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ca352a055eb44adb32da0787666021d in term 1.
I20260810 06:54:03.319769 18694 raft_consensus.cc:2454] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ca352a055eb44adb32da0787666021d in term 1.
I20260810 06:54:03.320044 18807 leader_election.cc:302] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25833405a9dc4e06a3d8a2a0b4a58ee2, 2ca352a055eb44adb32da0787666021d; no voters:
I20260810 06:54:03.320304 19047 raft_consensus.cc:2781] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:03.320755 19047 raft_consensus.cc:686] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 1 LEADER]: Becoming Leader. State: Replica: 2ca352a055eb44adb32da0787666021d, State: Running, Role: LEADER
I20260810 06:54:03.321271 19047 consensus_queue.cc:237] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } }
I20260810 06:54:03.325413 18251 catalog_manager.cc:5495] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ca352a055eb44adb32da0787666021d (127.22.11.131). New cstate: current_term: 1 leader_uuid: "2ca352a055eb44adb32da0787666021d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ca352a055eb44adb32da0787666021d" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 42077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 } health_report { overall_health: UNKNOWN } } }
I20260810 06:54:03.353798 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.348438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.22.11.131:42077)
0810 06:54:03.348659 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.348665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.348684 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:54:03.348904 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.348909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.348985 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.349118 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:54:03.349521 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.350222 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.351727 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.352149 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.352279 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.352765 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:54:03.353237 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:54:03.353386 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.353486 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.353564 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:54:03.361306 18483 raft_consensus.cc:1261] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Refusing update from remote peer 2ca352a055eb44adb32da0787666021d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:03.361304 18694 raft_consensus.cc:1261] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 1 FOLLOWER]: Refusing update from remote peer 2ca352a055eb44adb32da0787666021d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:03.362190 19054 consensus_queue.cc:1035] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd01c64ea13446e2812e9fd6dc2186dd" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:03.362509 19047 consensus_queue.cc:1035] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [LEADER]: Connected to new peer: Peer: permanent_uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:03.395185 18252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:03.395455 18252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:03.399176 18252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Sending DeleteTablet for 3 replicas of tablet 89dfc23a50754b83b866a98191a42b4c
I20260810 06:54:03.400516 18445 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dfc23a50754b83b866a98191a42b4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:55710
I20260810 06:54:03.400990 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 18309
I20260810 06:54:03.401279 18906 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dfc23a50754b83b866a98191a42b4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:57652
I20260810 06:54:03.401417 18660 tablet_service.cc:1514] Processing DeleteTablet for tablet 89dfc23a50754b83b866a98191a42b4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:03 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:55744
I20260810 06:54:03.402202 19086 tablet_replica.cc:330] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: stopping tablet replica
I20260810 06:54:03.402612 19086 raft_consensus.cc:2229] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:54:03.403249 19086 raft_consensus.cc:2258] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:03.405586 19086 ts_tablet_manager.cc:1905] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:03.410483 19084 tablet_replica.cc:330] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: stopping tablet replica
I20260810 06:54:03.410903 19084 raft_consensus.cc:2229] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:03.411451 19084 raft_consensus.cc:2258] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:03.411867 19090 tablet_replica.cc:330] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: stopping tablet replica
I20260810 06:54:03.412245 19090 raft_consensus.cc:2229] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:03.412806 19090 raft_consensus.cc:2258] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:03.414784 19090 ts_tablet_manager.cc:1905] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:03.414827 19084 ts_tablet_manager.cc:1905] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:03.416472 19086 ts_tablet_manager.cc:1918] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:03.416667 19086 log.cc:1196] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/89dfc23a50754b83b866a98191a42b4c
I20260810 06:54:03.417559 19086 ts_tablet_manager.cc:1939] T 89dfc23a50754b83b866a98191a42b4c P 2ca352a055eb44adb32da0787666021d: Deleting consensus metadata
I20260810 06:54:03.420571 18224 catalog_manager.cc:4841] TS 2ca352a055eb44adb32da0787666021d (127.22.11.131:42077): tablet 89dfc23a50754b83b866a98191a42b4c (table test-table [id=e29914bd4faa478e9bb49560802fb412]) successfully deleted
I20260810 06:54:03.422646 19038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.402975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.22.11.129:41379)
0810 06:54:03.403162 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.403168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.403187 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:54:03.403247 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.403252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.403330 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.403424 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:03.405895 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.407216 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.409529 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.410256 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.410380 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.415030 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:03.415042 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:03.415057 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:03.415102 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:03.418901 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.420003 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.420016 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.420020 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.420273 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.420861 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.420866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.420869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.421346 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:03.421644 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.422010 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.422259 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:54:03.422358 19084 ts_tablet_manager.cc:1918] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:03.423319 19084 log.cc:1196] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/89dfc23a50754b83b866a98191a42b4c
I20260810 06:54:03.423411 18542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:03.429283 19090 ts_tablet_manager.cc:1918] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:03.429448 19090 log.cc:1196] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/89dfc23a50754b83b866a98191a42b4c
I20260810 06:54:03.429952 19090 ts_tablet_manager.cc:1939] T 89dfc23a50754b83b866a98191a42b4c P cd01c64ea13446e2812e9fd6dc2186dd: Deleting consensus metadata
I20260810 06:54:03.431869 18223 catalog_manager.cc:4841] TS cd01c64ea13446e2812e9fd6dc2186dd (127.22.11.130:43521): tablet 89dfc23a50754b83b866a98191a42b4c (table test-table [id=e29914bd4faa478e9bb49560802fb412]) successfully deleted
I20260810 06:54:03.513551 19084 ts_tablet_manager.cc:1939] T 89dfc23a50754b83b866a98191a42b4c P 25833405a9dc4e06a3d8a2a0b4a58ee2: Deleting consensus metadata
I20260810 06:54:03.514358 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 18309
W20260810 06:54:03.539033 18223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.11.129:41379 (error 108)
I20260810 06:54:03.539222 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 18557
I20260810 06:54:03.541065 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.540609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55726)
0810 06:54:03.540792 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.540901 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:54:03.541342 18223 catalog_manager.cc:4584] TS 25833405a9dc4e06a3d8a2a0b4a58ee2 (127.22.11.129:41379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 89dfc23a50754b83b866a98191a42b4c: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
I20260810 06:54:03.555498 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.540373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.22.11.130:43521)
0810 06:54:03.540585 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.540592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.540631 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:54:03.540720 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.540726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.540807 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.540917 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:03.542438 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.543576 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.544997 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.545629 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.545760 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.548444 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:03.548463 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:03.548483 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:03.548526 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:03.552294 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.553299 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.553309 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.553313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.553501 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.553847 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.553851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.553854 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.554408 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:03.554720 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.554883 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.555117 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:54:03.556151 18772 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:03.595031 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.594435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55732)
0810 06:54:03.594698 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.594824 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 06:54:03.596314 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.595850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55740)
0810 06:54:03.596061 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.596158 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 06:54:03.645336 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 18557
I20260810 06:54:03.657205 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.656591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55754)
0810 06:54:03.656864 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.656986 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 06:54:03.658464 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.658031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55762)
0810 06:54:03.658204 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.658300 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 06:54:03.672019 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 18785
I20260810 06:54:03.686133 19035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.672929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57664 (local address 127.22.11.131:42077)
0810 06:54:03.673139 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:03.673145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:03.673171 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:54:03.673240 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:03.673245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:03.673310 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:03.673411 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:03.674795 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.675610 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.677065 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:03.677549 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:03.677669 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:03.680256 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:03.680272 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:03.680289 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:03.680337 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:03.683338 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.683996 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.684004 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.684006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.684174 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:03.684557 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:03.684564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:03.684566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:03.684969 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:03.685290 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:03.685566 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 06:54:03.685775 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 06:54:03.686817 19004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:03.739024 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.738527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55772)
0810 06:54:03.738726 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.738821 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:54:03.740144 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.739721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55776)
0810 06:54:03.739953 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.740029 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 06:54:03.811162 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 18785
I20260810 06:54:03.839097 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 18185
I20260810 06:54:03.840263 18297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:03.953778 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 18185
I20260810 06:54:03.953843 19037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:03.953089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:41379 (local address 127.0.0.1:55788)
0810 06:54:03.953541 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:03.953638 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:41379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311}
I20260810 06:54:23.983261 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34089
--webserver_interface=127.22.11.190
--webserver_port=42183
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:24.324290 19101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:24.324579 19101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:24.324725 19101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:24.331964 19101 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 06:54:24.332077 19101 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 06:54:24.332144 19101 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 06:54:24.332221 19101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:54:24.332296 19101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:24.332391 19101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:24.332471 19101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:54:24.332547 19101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:54:24.342293 19101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=42183
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:24.343478 19101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:24.345064 19101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:24.357136 19111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:24.357084 19117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:24.357160 19112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:24.360180 19101 server_base.cc:1034] running on GCE node
I20260810 06:54:24.360915 19101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:24.362149 19101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:24.363358 19101 hybrid_clock.cc:648] HybridClock initialized: now 1786344864363297 us; error 36 us; skew 500 ppm
I20260810 06:54:24.363662 19101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:24 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:24.366707 19101 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:54:24.368041 19101 webserver.cc:460] Webserver started at http://127.22.11.190:42183/ using document root <none> and password file <none>
I20260810 06:54:24.368721 19101 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:24.368829 19101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:24.373915 19101 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260810 06:54:24.376576 19131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:24.377830 19101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:54:24.377979 19101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "e6ec754c1b704d3eb650c0468629e0ad"
format_stamp: "Formatted at 2026-08-10 06:53:38 on dist-test-slave-f7th"
server_key: "ae74fdfbb5c1d0dea2e2dc15e4a83b94"
server_key_iv: "13a0df2eebecfe6e8a2057aa7800416a"
server_key_version: "encryptionkey@0"
I20260810 06:54:24.378865 19101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:24.400743 19101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:24.404305 19101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:24.404558 19101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:24.422127 19101 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34089
I20260810 06:54:24.422173 19209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34089 every 8 connection(s)
I20260810 06:54:24.423513 19101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:54:24.430227 19211 sys_catalog.cc:263] Verifying existing consensus state
I20260810 06:54:24.433358 19211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Bootstrap starting.
I20260810 06:54:24.433835 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 19101
I20260810 06:54:24.434628 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:41379
--local_ip_for_outbound_sockets=127.22.11.129
--tserver_master_addrs=127.22.11.190:34089
--webserver_port=36855
--webserver_interface=127.22.11.129
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:54:24.453647 19211 log.cc:824] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:24.469484 19211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 06:54:24.469981 19211 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: Bootstrap complete.
I20260810 06:54:24.478749 19211 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:24.479601 19211 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ec754c1b704d3eb650c0468629e0ad, State: Initialized, Role: FOLLOWER
I20260810 06:54:24.480163 19211 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:24.480373 19211 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:54:24.480477 19211 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:54:24.480667 19211 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 2 FOLLOWER]: Advancing to term 3
I20260810 06:54:24.484534 19211 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:24.485019 19211 leader_election.cc:302] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6ec754c1b704d3eb650c0468629e0ad; no voters:
I20260810 06:54:24.485618 19211 leader_election.cc:288] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 06:54:24.485755 19220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 3 FOLLOWER]: Leader election won for term 3
I20260810 06:54:24.487702 19211 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:54:24.492699 19220 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [term 3 LEADER]: Becoming Leader. State: Replica: e6ec754c1b704d3eb650c0468629e0ad, State: Running, Role: LEADER
I20260810 06:54:24.493278 19220 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } }
I20260810 06:54:24.495754 19223 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e6ec754c1b704d3eb650c0468629e0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } } }
I20260810 06:54:24.496104 19223 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:24.495777 19224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6ec754c1b704d3eb650c0468629e0ad. Latest consensus state: current_term: 3 leader_uuid: "e6ec754c1b704d3eb650c0468629e0ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6ec754c1b704d3eb650c0468629e0ad" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34089 } } }
I20260810 06:54:24.501508 19224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:24.511143 19245 catalog_manager.cc:1259] Loaded cluster ID: cf218a0daf8d42e49b832f82da676b0e
I20260810 06:54:24.511255 19245 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: loading cluster ID for follower catalog manager: success
I20260810 06:54:24.515216 19245 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e6ec754c1b704d3eb650c0468629e0ad: acquiring CA information for follower catalog manager: success
F20260810 06:54:24.517562 19245 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f43165769ad google::LogMessage::Fail() at ??:0
@ 0x7f431657988a google::LogMessage::SendToLog() at ??:0
@ 0x7f43165764de google::LogMessage::Flush() at ??:0
@ 0x7f4316577a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f431a2da7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f432bfabbbf kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f432bfab17b kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f432bf9a247 kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f432bf93c56 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f432c02e00c kudu::master::CatalogManagerBgTasks::Init()::$_0::operator()() at ??:0
@ 0x7f432c02de89 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f43191a0eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f431c47a6db start_thread at ??:0
@ 0x7f431541061f clone at ??:0
W20260810 06:54:24.824216 19216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:24.824494 19216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:24.824591 19216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:24.831761 19216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:24.831868 19216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:24.831992 19216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:54:24.841492 19216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:41379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=36855
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:24.842733 19216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:24.844014 19216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:24.856999 19281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:24.857111 19287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:24.857503 19279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:24.858119 19216 server_base.cc:1034] running on GCE node
I20260810 06:54:24.858696 19216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:24.859596 19216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:24.860786 19216 hybrid_clock.cc:648] HybridClock initialized: now 1786344864860734 us; error 41 us; skew 500 ppm
I20260810 06:54:24.861095 19216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:24 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344864, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:24.864362 19216 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:54:24.865885 19216 webserver.cc:460] Webserver started at http://127.22.11.129:36855/ using document root <none> and password file <none>
I20260810 06:54:24.866565 19216 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:24.866690 19216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:24.871970 19216 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:54:24.874668 19303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:24.875957 19216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:54:24.876124 19216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "25833405a9dc4e06a3d8a2a0b4a58ee2"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "048331157653467277d8aa55b6cdbcd4"
server_key_iv: "53d4863fa0b3c2af359228780df03dba"
server_key_version: "encryptionkey@0"
I20260810 06:54:24.877183 19216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:24.892588 19216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:24.896808 19216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:24.897076 19216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:24.898078 19216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:24.899461 19216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:24.899560 19216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:24.899672 19216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:24.899749 19216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:24.931620 19216 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:41379
I20260810 06:54:24.931701 19493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:41379 every 8 connection(s)
I20260810 06:54:24.933127 19216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:54:24.934546 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 19216
I20260810 06:54:24.935375 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:43521
--local_ip_for_outbound_sockets=127.22.11.130
--tserver_master_addrs=127.22.11.190:34089
--webserver_port=33167
--webserver_interface=127.22.11.130
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:54:24.938932 19496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:24.936577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:41307)
0810 06:54:24.937512 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:24.937792 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":340,"threads_started":1}
W20260810 06:54:24.939677 19494 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:34089 (0 consecutive failures): Network error: Failed to ping master at 127.22.11.190:34089: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
I20260810 06:54:24.940554 19496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:24.940200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:44257)
0810 06:54:24.940356 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:24.940434 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260810 06:54:24.941396 19496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:24.941099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:37855)
0810 06:54:24.941218 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:24.941292 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:54:24.941615 19494 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 06:54:25.268692 19498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:25.268956 19498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:25.269106 19498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:25.276082 19498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:25.276186 19498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:25.276345 19498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:54:25.285876 19498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:43521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=33167
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:25.287132 19498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:25.288388 19498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:25.300817 19505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:25.302066 19498 server_base.cc:1034] running on GCE node
W20260810 06:54:25.307097 19507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:25.307458 19510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:25.308233 19498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:25.309338 19498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:25.310552 19498 hybrid_clock.cc:648] HybridClock initialized: now 1786344865310499 us; error 41 us; skew 500 ppm
I20260810 06:54:25.310851 19498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:25 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:25.313870 19498 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:54:25.315251 19498 webserver.cc:460] Webserver started at http://127.22.11.130:33167/ using document root <none> and password file <none>
I20260810 06:54:25.315902 19498 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:25.316015 19498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:25.321270 19498 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:54:25.323932 19523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:25.325192 19498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:54:25.325343 19498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "cd01c64ea13446e2812e9fd6dc2186dd"
format_stamp: "Formatted at 2026-08-10 06:53:39 on dist-test-slave-f7th"
server_key: "8f6fd2e650fcd5c3dc1f0c8957d2f4f8"
server_key_iv: "d41ad306c3a9962ec101f7347fba3607"
server_key_version: "encryptionkey@0"
I20260810 06:54:25.326277 19498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:25.346163 19498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:25.349695 19498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:25.349924 19498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:25.350997 19498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:25.352731 19498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:25.352835 19498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:25.352949 19498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:25.353026 19498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:25.383399 19498 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:43521
I20260810 06:54:25.383461 19711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:43521 every 8 connection(s)
I20260810 06:54:25.384897 19498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:54:25.389381 19714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:25.387605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:43923)
0810 06:54:25.388290 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:25.388516 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":259,"threads_started":1}
W20260810 06:54:25.389959 19712 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:34089 (0 consecutive failures): Network error: Failed to ping master at 127.22.11.190:34089: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
I20260810 06:54:25.390647 19714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:25.390370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:51347)
0810 06:54:25.390486 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:25.390548 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:54:25.391429 19714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:25.391119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:48727)
0810 06:54:25.391251 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:25.391326 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:54:25.391652 19712 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:54:25.394093 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 19498
I20260810 06:54:25.394708 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:42077
--local_ip_for_outbound_sockets=127.22.11.131
--tserver_master_addrs=127.22.11.190:34089
--webserver_port=38047
--webserver_interface=127.22.11.131
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:25.722275 19717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:25.722528 19717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:25.722630 19717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:25.729598 19717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:25.729689 19717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:25.729803 19717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:54:25.739130 19717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:42077
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=38047
--tserver_master_addrs=127.22.11.190:34089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:25.740262 19717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:25.741526 19717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:25.753888 19730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:25.753949 19736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:25.753980 19732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:25.754724 19717 server_base.cc:1034] running on GCE node
I20260810 06:54:25.755290 19717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:25.756108 19717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:25.757305 19717 hybrid_clock.cc:648] HybridClock initialized: now 1786344865757251 us; error 40 us; skew 500 ppm
I20260810 06:54:25.757594 19717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:25 dist-test-slave-f7th krb5kdc[17251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344865, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:25.763388 19717 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:54:25.764786 19717 webserver.cc:460] Webserver started at http://127.22.11.131:38047/ using document root <none> and password file <none>
I20260810 06:54:25.765396 19717 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:25.765520 19717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:25.770474 19717 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 06:54:25.772951 19745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:25.774113 19717 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:54:25.774255 19717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "2ca352a055eb44adb32da0787666021d"
format_stamp: "Formatted at 2026-08-10 06:53:40 on dist-test-slave-f7th"
server_key: "c92ea535e85c4d1072ca364cec18c89f"
server_key_iv: "c980915739e7a4700b26b81cc371a94a"
server_key_version: "encryptionkey@0"
I20260810 06:54:25.775139 19717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:25.787184 19717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:25.790573 19717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:25.790809 19717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:25.791643 19717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:25.792866 19717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:25.792960 19717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:25.793061 19717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:25.793131 19717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:25.823757 19717 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:42077
I20260810 06:54:25.823836 19948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:42077 every 8 connection(s)
I20260810 06:54:25.825196 19717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:54:25.829739 19952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:25.827899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:60435)
0810 06:54:25.828583 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:25.828848 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":258,"threads_started":1}
W20260810 06:54:25.830333 19949 heartbeater.cc:644] Failed to heartbeat to 127.22.11.190:34089 (0 consecutive failures): Network error: Failed to ping master at 127.22.11.190:34089: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
I20260810 06:54:25.831099 19952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:25.830807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:44177)
0810 06:54:25.830929 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:25.830991 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:54:25.831754 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 19717
I20260810 06:54:25.831914 19952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:25.831552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:58211)
0810 06:54:25.831695 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:25.831783 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 06:54:25.832124 19949 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 06:54:25.942855 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:25.942085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:50559)
0810 06:54:25.942534 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:25.942638 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":221,"threads_started":1}
I20260810 06:54:26.392920 19959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:26.392137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:52955)
0810 06:54:26.392563 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:26.392695 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":209,"threads_started":1}
I20260810 06:54:26.833426 19960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:26.832604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:59581)
0810 06:54:26.833089 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:26.833199 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":241,"threads_started":1}
I20260810 06:54:26.944296 19961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:26.943632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:59261)
0810 06:54:26.944062 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:26.944141 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":212,"threads_started":1}
I20260810 06:54:27.394356 19963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:27.393677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:57797)
0810 06:54:27.394124 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:27.394202 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":221,"threads_started":1}
I20260810 06:54:27.834983 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:27.834216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:58151)
0810 06:54:27.834714 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:27.834801 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":264,"threads_started":1}
I20260810 06:54:27.945847 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:27.945127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:41559)
0810 06:54:27.945585 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:27.945673 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":224,"threads_started":1}
I20260810 06:54:28.395803 19967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:28.395128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:48653)
0810 06:54:28.395560 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:28.395638 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":211,"threads_started":1}
I20260810 06:54:28.836498 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:28.835783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:56545)
0810 06:54:28.836254 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:28.836331 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":271,"threads_started":1}
I20260810 06:54:28.947376 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:28.946666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:42199)
0810 06:54:28.947109 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:28.947188 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":231,"threads_started":1}
I20260810 06:54:29.397384 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:29.396575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:36899)
0810 06:54:29.397064 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:29.397143 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":250,"threads_started":1}
I20260810 06:54:29.838035 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:29.837299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:51021)
0810 06:54:29.837750 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:29.837834 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":236,"threads_started":1}
I20260810 06:54:29.948890 19977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:29.948181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:49599)
0810 06:54:29.948642 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:29.948725 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":220,"threads_started":1}
I20260810 06:54:30.398924 19978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:30.398213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:39871)
0810 06:54:30.398684 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:30.398765 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":228,"threads_started":1}
I20260810 06:54:30.839562 19979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:30.838855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:38527)
0810 06:54:30.839295 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:30.839376 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":236,"threads_started":1}
I20260810 06:54:30.950348 19980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:30.949692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:60969)
0810 06:54:30.950111 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:30.950197 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":217,"threads_started":1}
I20260810 06:54:31.400359 19982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:31.399690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:43723)
0810 06:54:31.400118 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:31.400207 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":208,"threads_started":1}
I20260810 06:54:31.841074 19987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:31.840358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:54259)
0810 06:54:31.840819 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:31.840908 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":256,"threads_started":1}
I20260810 06:54:31.951934 19988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:31.951146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:58217)
0810 06:54:31.951646 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:31.951725 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":222,"threads_started":1}
I20260810 06:54:32.401968 19989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:32.401190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:58149)
0810 06:54:32.401673 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:32.401754 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":200,"threads_started":1}
I20260810 06:54:32.842622 19990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:32.841870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:56715)
0810 06:54:32.842328 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:32.842406 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":211,"threads_started":1}
I20260810 06:54:32.953512 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:32.952786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:35317)
0810 06:54:32.953262 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:32.953342 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":252,"threads_started":1}
I20260810 06:54:33.403469 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:33.402780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:35789)
0810 06:54:33.403214 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:33.403294 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":214,"threads_started":1}
I20260810 06:54:33.844081 19997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:33.843382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:45723)
0810 06:54:33.843829 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:33.843908 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
I20260810 06:54:33.955067 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:33.954297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:54455)
0810 06:54:33.954818 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:33.954894 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":280,"threads_started":1}
I20260810 06:54:34.405001 20000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:34.404253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:51661)
0810 06:54:34.404734 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:34.404813 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":228,"threads_started":1}
I20260810 06:54:34.845649 20001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:34.844912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:47363)
0810 06:54:34.845369 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:34.845447 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":226,"threads_started":1}
I20260810 06:54:34.956549 20002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:34.955903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:45901)
0810 06:54:34.956310 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:34.956389 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":220,"threads_started":1}
I20260810 06:54:35.406488 20003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:35.405816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:34709)
0810 06:54:35.406249 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:35.406328 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260810 06:54:35.847184 20004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:35.846465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:39405)
0810 06:54:35.846930 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:35.847026 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":266,"threads_started":1}
I20260810 06:54:35.958153 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:35.957451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:37783)
0810 06:54:35.957874 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:35.957973 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":209,"threads_started":1}
I20260810 06:54:36.408021 20006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:36.407290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:52241)
0810 06:54:36.407754 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:36.407845 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":235,"threads_started":1}
I20260810 06:54:36.848878 20007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:36.848020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:37623)
0810 06:54:36.848572 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:36.848690 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":269,"threads_started":1}
I20260810 06:54:36.959759 20008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:36.959038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:45621)
0810 06:54:36.959524 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:36.959603 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
I20260810 06:54:37.409550 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:37.408873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:40937)
0810 06:54:37.409310 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:37.409391 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":228,"threads_started":1}
I20260810 06:54:37.850427 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:37.849708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:34631)
0810 06:54:37.850153 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:37.850239 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":222,"threads_started":1}
I20260810 06:54:37.961470 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:37.960648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:47757)
0810 06:54:37.961177 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:37.961274 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":275,"threads_started":1}
I20260810 06:54:38.411142 20017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:38.410380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:60879)
0810 06:54:38.410868 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:38.410960 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":247,"threads_started":1}
I20260810 06:54:38.852095 20021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:38.851309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:34143)
0810 06:54:38.851820 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:38.851915 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":268,"threads_started":1}
I20260810 06:54:38.962981 20023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:38.962319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:48511)
0810 06:54:38.962719 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:38.962806 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":204,"threads_started":1}
I20260810 06:54:39.412735 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:39.412006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:52777)
0810 06:54:39.412418 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:39.412517 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":202,"threads_started":1}
I20260810 06:54:39.853664 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:39.852961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:43283)
0810 06:54:39.853395 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:39.853477 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":219,"threads_started":1}
I20260810 06:54:39.964704 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:39.963991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.129:36345)
0810 06:54:39.964402 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:39.964488 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":210,"threads_started":1}
I20260810 06:54:40.414548 20029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:40.413772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.130:38571)
0810 06:54:40.414243 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:40.414349 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":249,"threads_started":1}
I20260810 06:54:40.834944 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 19216
I20260810 06:54:40.852031 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:40.836096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57836 (local address 127.22.11.129:41379)
0810 06:54:40.836750 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:40.836765 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:40.836805 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:54:40.838409 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:40.838419 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:40.838846 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:40.839141 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:40.840503 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:40.841796 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:40.843026 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:40.843665 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:40.843796 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:40.846374 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:40.846382 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:40.846398 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:40.846434 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:40.849185 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:40.849787 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:40.849793 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:40.849799 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:40.849967 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:40.850342 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:40.850345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:40.850347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:40.850866 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:40.851111 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:40.851418 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 06:54:40.851741 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":302,"threads_started":1}
I20260810 06:54:40.853425 19487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:40.855561 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:40.854689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34089 (local address 127.22.11.131:46105)
0810 06:54:40.855193 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:40.855323 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.190:34089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":231,"threads_started":1}
I20260810 06:54:40.937773 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 19216
I20260810 06:54:40.971450 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 19498
I20260810 06:54:40.988222 20039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:40.972331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.22.11.130:43521)
0810 06:54:40.972917 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:40.972931 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:40.972970 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:54:40.974559 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:40.974569 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:40.974981 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:40.975265 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:40.976726 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:40.978061 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:40.979474 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:40.980132 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:40.980265 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:40.982733 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:40.982741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:40.982755 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:40.982790 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:40.985583 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:40.986151 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:40.986156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:40.986162 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:40.986318 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:40.986578 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:40.986581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:40.986583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:40.987064 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:40.987287 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:40.987575 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 06:54:40.987941 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":302,"threads_started":1}
I20260810 06:54:40.989529 19706 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:41.072263 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 19498
I20260810 06:54:41.097363 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 19717
I20260810 06:54:41.114157 20041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:41.098467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59264 (local address 127.22.11.131:42077)
0810 06:54:41.099062 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:41.099076 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:41.099118 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:54:41.100801 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:41.100812 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:41.101247 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:41.101526 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:41.102800 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:41.103977 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:41.105339 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:41.105959 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:41.106110 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:41.108556 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:41.108563 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:41.108577 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:41.108638 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:41.111247 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:41.111862 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:41.111868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:41.111874 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:41.112044 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:41.112324 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:41.112327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:41.112329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:41.112872 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:41.113146 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:41.113496 (+ 350us) server_negotiation.cc:299] Negotiation successful
0810 06:54:41.113823 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":262,"threads_started":1}
I20260810 06:54:41.115479 19943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:41.197602 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 19717
2026-08-10T06:54:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64497 ms)
[----------] 30 tests from SecurityITest (301043 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:54:41 dist-test-slave-f7th krb5kdc[20049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:54:41 dist-test-slave-f7th krb5kdc[20049](info): set up 2 sockets
Aug 10 06:54:41 dist-test-slave-f7th krb5kdc[20049](info): commencing operation
krb5kdc: starting...
W20260810 06:54:42.454969 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:54:42 dist-test-slave-f7th krb5kdc[20049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:54:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:54:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:54:42.729561 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:42141
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:42141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:43.062592 20076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:43.062932 20076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:43.063088 20076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:43.070320 20076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:54:43.070436 20076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:43.070519 20076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:43.070595 20076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:54:43.070657 20076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:54:43.080327 20076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:42141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:42141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:43.081655 20076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:43.083003 20076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:43.094769 20091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:43.094842 20088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:43.094803 20087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:43.095430 20076 server_base.cc:1034] running on GCE node
I20260810 06:54:43.096233 20076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:43.097646 20076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:43.098903 20076 hybrid_clock.cc:648] HybridClock initialized: now 1786344883098806 us; error 54 us; skew 500 ppm
I20260810 06:54:43.099246 20076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:43 dist-test-slave-f7th krb5kdc[20049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:43.102335 20076 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:54:43.103788 20076 webserver.cc:460] Webserver started at http://127.22.11.190:39381/ using document root <none> and password file <none>
I20260810 06:54:43.104461 20076 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:43.104564 20076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:43.104884 20076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:43.107831 20076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "ee3f720b996a47a1a8156e1062a988d4"
format_stamp: "Formatted at 2026-08-10 06:54:43 on dist-test-slave-f7th"
server_key: "cf9947a1830882b3814ede89a70e0aa8"
server_key_iv: "2e0c7298b6a9dc1f583565433a7ceea6"
server_key_version: "encryptionkey@0"
I20260810 06:54:43.108547 20076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "ee3f720b996a47a1a8156e1062a988d4"
format_stamp: "Formatted at 2026-08-10 06:54:43 on dist-test-slave-f7th"
server_key: "cf9947a1830882b3814ede89a70e0aa8"
server_key_iv: "2e0c7298b6a9dc1f583565433a7ceea6"
server_key_version: "encryptionkey@0"
I20260810 06:54:43.113631 20076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 06:54:43.116886 20105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:43.118114 20076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:54:43.118268 20076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "ee3f720b996a47a1a8156e1062a988d4"
format_stamp: "Formatted at 2026-08-10 06:54:43 on dist-test-slave-f7th"
server_key: "cf9947a1830882b3814ede89a70e0aa8"
server_key_iv: "2e0c7298b6a9dc1f583565433a7ceea6"
server_key_version: "encryptionkey@0"
I20260810 06:54:43.118448 20076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:43.131314 20076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:43.134936 20076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:43.135186 20076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:43.152017 20076 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:42141
I20260810 06:54:43.152036 20192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:42141 every 8 connection(s)
I20260810 06:54:43.153374 20076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:54:43.153927 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 20076
I20260810 06:54:43.154178 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:54:43.155038 22574 external_mini_cluster.cc:1428] Setting key e5b36d8ba922a899ab64f4a38d242082
I20260810 06:54:43.157943 20193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:54:43 dist-test-slave-f7th krb5kdc[20049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:43.170691 20193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4: Bootstrap starting.
I20260810 06:54:43.173825 20193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:43.175064 20193 log.cc:824] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:43.177858 20200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:43.157126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53928 (local address 127.22.11.190:42141)
0810 06:54:43.157862 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:43.157874 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:43.157913 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:54:43.159582 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:43.159602 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:43.159937 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:43.160265 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:43.162086 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:43.163405 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:43.165316 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:43.165963 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:43.166093 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:43.171066 (+ 4973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:43.171073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:43.171087 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:43.171126 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:43.173949 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:43.174780 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:43.174786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:43.174791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:43.174941 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:43.175322 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:43.175326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:43.175328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:43.175967 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:43.176276 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:43.176555 (+ 279us) server_negotiation.cc:299] Negotiation successful
0810 06:54:43.176930 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":351,"threads_started":1}
I20260810 06:54:43.178072 20193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4: No bootstrap required, opened a new log
I20260810 06:54:43.185586 20193 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3f720b996a47a1a8156e1062a988d4" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42141 } }
I20260810 06:54:43.185902 20193 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:43.185998 20193 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee3f720b996a47a1a8156e1062a988d4, State: Initialized, Role: FOLLOWER
I20260810 06:54:43.186568 20193 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3f720b996a47a1a8156e1062a988d4" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42141 } }
I20260810 06:54:43.186848 20193 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:54:43.186998 20193 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:54:43.187157 20193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:43.189204 20193 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3f720b996a47a1a8156e1062a988d4" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42141 } }
I20260810 06:54:43.189639 20193 leader_election.cc:302] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee3f720b996a47a1a8156e1062a988d4; no voters:
I20260810 06:54:43.190171 20193 leader_election.cc:288] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:54:43.190307 20202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:43.191315 20202 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [term 1 LEADER]: Becoming Leader. State: Replica: ee3f720b996a47a1a8156e1062a988d4, State: Running, Role: LEADER
I20260810 06:54:43.191782 20202 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3f720b996a47a1a8156e1062a988d4" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42141 } }
I20260810 06:54:43.192099 20193 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:54:43.194463 20206 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee3f720b996a47a1a8156e1062a988d4. Latest consensus state: current_term: 1 leader_uuid: "ee3f720b996a47a1a8156e1062a988d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3f720b996a47a1a8156e1062a988d4" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42141 } } }
I20260810 06:54:43.195061 20206 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:43.195549 20204 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee3f720b996a47a1a8156e1062a988d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee3f720b996a47a1a8156e1062a988d4" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 42141 } } }
I20260810 06:54:43.195880 20204 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4 [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:43.196826 20216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:54:43.201663 20216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:54:43.209793 20216 catalog_manager.cc:1347] Generated new cluster ID: 7f2d728f0939437b89b30a5c18dbdba4
I20260810 06:54:43.209892 20216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:54:43.225845 20216 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:54:43.227609 20216 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:54:43.241432 20216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4: Generated new TSK 0
I20260810 06:54:43.242367 20216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:54:43.311760 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42141
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:43.645627 20247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:43.645931 20247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:43.646070 20247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:43.653419 20247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:43.653551 20247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:43.653703 20247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:54:43.663605 20247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:43.664983 20247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:43.666324 20247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:43.679279 20260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:43.679318 20255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:43.679608 20254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:43.680039 20247 server_base.cc:1034] running on GCE node
I20260810 06:54:43.680603 20247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:43.681528 20247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:43.682731 20247 hybrid_clock.cc:648] HybridClock initialized: now 1786344883682690 us; error 28 us; skew 500 ppm
I20260810 06:54:43.683025 20247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:43 dist-test-slave-f7th krb5kdc[20049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:43.686190 20247 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:54:43.687603 20247 webserver.cc:460] Webserver started at http://127.22.11.129:33961/ using document root <none> and password file <none>
I20260810 06:54:43.688235 20247 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:43.688337 20247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:43.688602 20247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:43.691525 20247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "1191a60f4adb43e8b628408fcff20284"
format_stamp: "Formatted at 2026-08-10 06:54:43 on dist-test-slave-f7th"
server_key: "2d493a1efdba608e6721e8970af7b90a"
server_key_iv: "11dbc1f24c5228b0ef5365c595b10178"
server_key_version: "encryptionkey@0"
I20260810 06:54:43.692236 20247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "1191a60f4adb43e8b628408fcff20284"
format_stamp: "Formatted at 2026-08-10 06:54:43 on dist-test-slave-f7th"
server_key: "2d493a1efdba608e6721e8970af7b90a"
server_key_iv: "11dbc1f24c5228b0ef5365c595b10178"
server_key_version: "encryptionkey@0"
I20260810 06:54:43.697398 20247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 06:54:43.700737 20273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:43.701967 20247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:54:43.702112 20247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "1191a60f4adb43e8b628408fcff20284"
format_stamp: "Formatted at 2026-08-10 06:54:43 on dist-test-slave-f7th"
server_key: "2d493a1efdba608e6721e8970af7b90a"
server_key_iv: "11dbc1f24c5228b0ef5365c595b10178"
server_key_version: "encryptionkey@0"
I20260810 06:54:43.702299 20247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:43.724963 20247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:43.729195 20247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:43.729466 20247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:43.730468 20247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:43.731796 20247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:43.731875 20247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:43.731989 20247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:43.732066 20247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:43.762151 20247 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:38347
I20260810 06:54:43.762202 20476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:38347 every 8 connection(s)
I20260810 06:54:43.763438 20247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:54:43.769191 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 20247
I20260810 06:54:43.769394 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:54:43.770030 22574 external_mini_cluster.cc:1428] Setting key 07631034d7904aa44d0bc2bd20dd9320
Aug 10 06:54:43 dist-test-slave-f7th krb5kdc[20049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344883, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:43.782567 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:43.765974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:55355 (local address 127.22.11.190:42141)
0810 06:54:43.766409 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:43.766414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:43.767351 (+ 937us) server_negotiation.cc:407] Connection header received
0810 06:54:43.769252 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:43.769258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:43.769344 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:43.769452 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:43.771067 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:43.771800 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:43.773362 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:43.773759 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:43.773875 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:43.777190 (+ 3315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:43.777195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:43.777198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:43.777231 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:43.779598 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:43.780282 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:43.780286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:43.780289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:43.780410 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:43.780860 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:43.780864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:43.780866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:43.781140 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:43.781290 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:43.781988 (+ 698us) server_negotiation.cc:299] Negotiation successful
0810 06:54:43.782222 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":210,"threads_started":1}
I20260810 06:54:43.783512 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:43.766506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:42141 (local address 127.22.11.129:55355)
0810 06:54:43.767209 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:43.767245 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:43.768994 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:43.769697 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:43.769705 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:43.770224 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:43.770900 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:43.770922 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:43.771944 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:43.771948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:43.773226 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:43.773238 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:43.773979 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:43.773987 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:43.774202 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:43.774808 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:43.774830 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:43.776990 (+ 2160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:43.779727 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:43.779737 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:43.779753 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:43.780159 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:43.780518 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:43.780522 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:43.780526 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:43.780749 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:43.781267 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:43.781274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:43.781613 (+ 339us) client_negotiation.cc:769] Sending connection context
0810 06:54:43.781846 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 06:54:43.782344 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":280,"threads_started":1}
I20260810 06:54:43.785036 20478 heartbeater.cc:347] Connected to a master server at 127.22.11.190:42141
I20260810 06:54:43.785362 20478 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:43.786199 20478 heartbeater.cc:511] Master 127.22.11.190:42141 requested a full tablet report, sending...
I20260810 06:54:43.788334 20133 ts_manager.cc:194] Registered new tserver with Master: 1191a60f4adb43e8b628408fcff20284 (127.22.11.129:38347)
I20260810 06:54:43.789795 20133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:55355
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:54:43.835737 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42141
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:44.175685 20489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:44.175952 20489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:44.176090 20489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:44.183179 20489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:44.183286 20489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:44.183462 20489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:54:44.193079 20489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:44.194294 20489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:44.195602 20489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:44.208149 20502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:44.208199 20500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:44.208225 20497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:44.209183 20489 server_base.cc:1034] running on GCE node
I20260810 06:54:44.209764 20489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:44.210601 20489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:44.211795 20489 hybrid_clock.cc:648] HybridClock initialized: now 1786344884211745 us; error 42 us; skew 500 ppm
I20260810 06:54:44.212080 20489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:44 dist-test-slave-f7th krb5kdc[20049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:44.215098 20489 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:54:44.216482 20489 webserver.cc:460] Webserver started at http://127.22.11.130:44897/ using document root <none> and password file <none>
I20260810 06:54:44.217175 20489 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:44.217294 20489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:44.217593 20489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:44.220326 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "4ea00de5332b42e38cd97d7ed038d042"
format_stamp: "Formatted at 2026-08-10 06:54:44 on dist-test-slave-f7th"
server_key: "784c1ae252b79bf396dbec085f8806d5"
server_key_iv: "4281b13349190d21505b330a0b8add42"
server_key_version: "encryptionkey@0"
I20260810 06:54:44.221076 20489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "4ea00de5332b42e38cd97d7ed038d042"
format_stamp: "Formatted at 2026-08-10 06:54:44 on dist-test-slave-f7th"
server_key: "784c1ae252b79bf396dbec085f8806d5"
server_key_iv: "4281b13349190d21505b330a0b8add42"
server_key_version: "encryptionkey@0"
I20260810 06:54:44.225961 20489 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:54:44.229161 20512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:44.230346 20489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260810 06:54:44.230504 20489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "4ea00de5332b42e38cd97d7ed038d042"
format_stamp: "Formatted at 2026-08-10 06:54:44 on dist-test-slave-f7th"
server_key: "784c1ae252b79bf396dbec085f8806d5"
server_key_iv: "4281b13349190d21505b330a0b8add42"
server_key_version: "encryptionkey@0"
I20260810 06:54:44.230681 20489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:44.245062 20489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:44.248752 20489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:44.249015 20489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:44.249912 20489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:44.251138 20489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:44.251217 20489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:44.251336 20489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:44.251402 20489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:44.281903 20489 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:42049
I20260810 06:54:44.281944 20694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:42049 every 8 connection(s)
I20260810 06:54:44.283207 20489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:54:44.284637 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 20489
I20260810 06:54:44.284832 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:54:44.285315 22574 external_mini_cluster.cc:1428] Setting key 526630c8789db1d9bcf1c62275a22cff
Aug 10 06:54:44 dist-test-slave-f7th krb5kdc[20049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:44.306357 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.287431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:54175 (local address 127.22.11.190:42141)
0810 06:54:44.287912 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.287918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.289125 (+ 1207us) server_negotiation.cc:407] Connection header received
0810 06:54:44.291414 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.291418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.291478 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.291576 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:44.293435 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.294179 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.296156 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.296590 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.296716 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.300461 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:44.300465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:44.300469 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:44.300512 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:44.303456 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:44.304123 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:44.304128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:44.304131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:44.304282 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:44.304672 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:44.304676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:44.304679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:44.304982 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:44.305122 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.305816 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.306020 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":246,"threads_started":1}
I20260810 06:54:44.307117 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.288107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:42141 (local address 127.22.11.130:54175)
0810 06:54:44.288943 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:44.288995 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:44.291156 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:44.291859 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:44.291869 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:44.292527 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:44.293284 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.293308 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.294331 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.294336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.296016 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.296031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.296814 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.296821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.297084 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.297728 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:44.297754 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:44.300248 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:44.303600 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:44.303606 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:44.303617 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:44.304004 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:44.304375 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:44.304378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:44.304380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:44.304546 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:44.305101 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:44.305106 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:44.305438 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 06:54:44.305692 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 06:54:44.306065 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":284,"threads_started":1}
I20260810 06:54:44.308424 20695 heartbeater.cc:347] Connected to a master server at 127.22.11.190:42141
I20260810 06:54:44.308732 20695 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:44.309409 20695 heartbeater.cc:511] Master 127.22.11.190:42141 requested a full tablet report, sending...
I20260810 06:54:44.310814 20133 ts_manager.cc:194] Registered new tserver with Master: 4ea00de5332b42e38cd97d7ed038d042 (127.22.11.130:42049)
I20260810 06:54:44.311604 20133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:54175
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:54:44.349664 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42141
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:44.677037 20704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:44.677295 20704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:44.677397 20704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:44.684383 20704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:44.684480 20704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:44.684597 20704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:54:44.694360 20704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:42141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:44.695562 20704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:44.696962 20704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:44.709709 20715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:44.709739 20716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:44.710052 20722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:44.710423 20704 server_base.cc:1034] running on GCE node
I20260810 06:54:44.711018 20704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:44.711920 20704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:44.713143 20704 hybrid_clock.cc:648] HybridClock initialized: now 1786344884713085 us; error 46 us; skew 500 ppm
I20260810 06:54:44.713454 20704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:44 dist-test-slave-f7th krb5kdc[20049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:44.716400 20704 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:54:44.717813 20704 webserver.cc:460] Webserver started at http://127.22.11.131:37509/ using document root <none> and password file <none>
I20260810 06:54:44.718461 20704 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:44.718583 20704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:44.718869 20704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:44.721712 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "8ec582ca422f4e66ac787f240c111e57"
format_stamp: "Formatted at 2026-08-10 06:54:44 on dist-test-slave-f7th"
server_key: "a4724cf2ae4e366fd7734909e9c58e45"
server_key_iv: "8c0ac9a53dcce8d5abe55313bdef49fd"
server_key_version: "encryptionkey@0"
I20260810 06:54:44.722445 20704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "8ec582ca422f4e66ac787f240c111e57"
format_stamp: "Formatted at 2026-08-10 06:54:44 on dist-test-slave-f7th"
server_key: "a4724cf2ae4e366fd7734909e9c58e45"
server_key_iv: "8c0ac9a53dcce8d5abe55313bdef49fd"
server_key_version: "encryptionkey@0"
I20260810 06:54:44.727341 20704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 06:54:44.730562 20734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:44.731736 20704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:54:44.731868 20704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "8ec582ca422f4e66ac787f240c111e57"
format_stamp: "Formatted at 2026-08-10 06:54:44 on dist-test-slave-f7th"
server_key: "a4724cf2ae4e366fd7734909e9c58e45"
server_key_iv: "8c0ac9a53dcce8d5abe55313bdef49fd"
server_key_version: "encryptionkey@0"
I20260810 06:54:44.732035 20704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:44.752818 20704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:44.756421 20704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:44.756672 20704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:44.757519 20704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:44.758719 20704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:44.758792 20704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:44.758880 20704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:44.758924 20704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:44.790503 20704 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:36985
I20260810 06:54:44.790570 20930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:36985 every 8 connection(s)
I20260810 06:54:44.791718 20704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:54:44.793134 20478 heartbeater.cc:503] Master 127.22.11.190:42141 was elected leader, sending a full tablet report...
I20260810 06:54:44.795215 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 20704
I20260810 06:54:44.795415 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:54:44.796038 22574 external_mini_cluster.cc:1428] Setting key 8e5866d884641c45fd596323c3efa46f
Aug 10 06:54:44 dist-test-slave-f7th krb5kdc[20049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344884, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:44.811499 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.794088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:53633 (local address 127.22.11.190:42141)
0810 06:54:44.794250 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.794255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.795845 (+ 1590us) server_negotiation.cc:407] Connection header received
0810 06:54:44.798081 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.798087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.798188 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.798269 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:44.799813 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.800536 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.802040 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.802442 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.802562 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.805737 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:44.805743 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:44.805748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:44.805787 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:44.808477 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:44.809165 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:44.809170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:44.809173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:44.809325 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:44.809807 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:44.809811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:44.809815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:44.810178 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:44.810322 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.811089 (+ 767us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.811262 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:54:44.812390 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.794809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:42141 (local address 127.22.11.131:53633)
0810 06:54:44.795679 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:44.795740 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:44.797819 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:44.798514 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:44.798523 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:44.799111 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:44.799652 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.799669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.800703 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.800706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.801900 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.801912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.802649 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.802656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.802866 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.803445 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:44.803467 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:44.805560 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:44.808604 (+ 3044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:44.808631 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:44.808647 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:44.809044 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:44.809439 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:44.809444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:44.809448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:44.809684 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:44.810304 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:44.810312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:44.810662 (+ 350us) client_negotiation.cc:769] Sending connection context
0810 06:54:44.810963 (+ 301us) client_negotiation.cc:240] Negotiation successful
0810 06:54:44.811340 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":324,"threads_started":1}
I20260810 06:54:44.813611 20931 heartbeater.cc:347] Connected to a master server at 127.22.11.190:42141
I20260810 06:54:44.813871 20931 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:44.814534 20931 heartbeater.cc:511] Master 127.22.11.190:42141 requested a full tablet report, sending...
I20260810 06:54:44.815785 20133 ts_manager.cc:194] Registered new tserver with Master: 8ec582ca422f4e66ac787f240c111e57 (127.22.11.131:36985)
I20260810 06:54:44.816527 20133 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:53633
I20260810 06:54:44.822751 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:54:44.838768 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.827123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.22.11.190:42141)
0810 06:54:44.827274 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.827278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.827739 (+ 461us) server_negotiation.cc:407] Connection header received
0810 06:54:44.827869 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.827873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.827946 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.828022 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:44.829392 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.830081 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.831306 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.831698 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.831782 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.834158 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:44.834162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:44.834166 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:44.834195 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:44.836641 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:44.837230 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:44.837234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:44.837237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:44.837360 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:44.837633 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:44.837636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:44.837638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:44.837922 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:44.838112 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.838377 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.838533 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:54:44.842622 20133 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:54:44.849466 20133 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:54:44.870865 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.861212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.22.11.130:42049)
0810 06:54:44.862112 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.862119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.862143 (+ 24us) server_negotiation.cc:407] Connection header received
0810 06:54:44.862304 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.862311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.862553 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.862754 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:44.863678 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.865724 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.868400 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.869695 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.869828 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.870017 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.870359 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.870525 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":626,"threads_started":1}
I20260810 06:54:44.870865 20949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.860952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:42049 (local address 127.0.0.1:36998)
0810 06:54:44.861887 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:44.861921 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:44.862098 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:44.863030 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:44.863037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:44.863064 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:44.863491 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.863506 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.865867 (+ 2361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.865872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.868239 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.868258 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.869896 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.869902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.870111 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.870164 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:54:44.870297 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 06:54:44.870483 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":593,"threads_started":1}
I20260810 06:54:44.871773 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.862314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:36985 (local address 127.0.0.1:45850)
0810 06:54:44.862828 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:44.862846 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:44.862956 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:44.864754 (+ 1798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:44.864758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:44.864785 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:44.865157 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.865172 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.867371 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.867377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.869331 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.869345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.870977 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.870984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.871170 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.871203 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:54:44.871257 (+ 54us) client_negotiation.cc:240] Negotiation successful
0810 06:54:44.871339 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":25,"thread_start_us":255,"threads_started":1}
I20260810 06:54:44.871910 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.862846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:38347 (local address 127.0.0.1:52180)
0810 06:54:44.863483 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:44.863513 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:44.863621 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:44.865047 (+ 1426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:44.865051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:44.865080 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:44.865484 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.865495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.866873 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.866877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.868958 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.868973 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.870837 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.870843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.871265 (+ 422us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.871299 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:54:44.871395 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:54:44.871524 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":48,"thread_start_us":847,"threads_started":1}
I20260810 06:54:44.872968 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.863506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52180 (local address 127.22.11.129:38347)
0810 06:54:44.864459 (+ 953us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.864468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.864494 (+ 26us) server_negotiation.cc:407] Connection header received
0810 06:54:44.864608 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.864639 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.864840 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.865042 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:44.865637 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.866726 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.869115 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.870370 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.870489 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.870692 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.872469 (+ 1777us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.872657 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":663,"threads_started":1}
I20260810 06:54:44.872978 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.862714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.22.11.131:36985)
0810 06:54:44.864154 (+ 1440us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.864162 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.864193 (+ 31us) server_negotiation.cc:407] Connection header received
0810 06:54:44.864283 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.864290 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.864524 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.865382 (+ 858us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:44.865684 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.867221 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.869497 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.870724 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.872185 (+ 1461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.872411 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.872471 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.872648 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1274,"thread_start_us":1108,"threads_started":1}
I20260810 06:54:44.874800 20591 tablet_service.cc:1467] Processing CreateTablet for tablet 50c8965bdecf489b82e85103ce43c761 (DEFAULT_TABLE table=test-table [id=fbd177f58bbe41198179e0ff62cabb3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:44.876092 20591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c8965bdecf489b82e85103ce43c761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:44.876338 20828 tablet_service.cc:1467] Processing CreateTablet for tablet 50c8965bdecf489b82e85103ce43c761 (DEFAULT_TABLE table=test-table [id=fbd177f58bbe41198179e0ff62cabb3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:44.876708 20358 tablet_service.cc:1467] Processing CreateTablet for tablet 50c8965bdecf489b82e85103ce43c761 (DEFAULT_TABLE table=test-table [id=fbd177f58bbe41198179e0ff62cabb3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:44.877810 20828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c8965bdecf489b82e85103ce43c761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:44.877986 20358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50c8965bdecf489b82e85103ce43c761. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:44.886895 20959 tablet_bootstrap.cc:492] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Bootstrap starting.
I20260810 06:54:44.890452 20959 tablet_bootstrap.cc:654] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:44.891536 20960 tablet_bootstrap.cc:492] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Bootstrap starting.
I20260810 06:54:44.891640 20959 log.cc:824] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:44.893838 20961 tablet_bootstrap.cc:492] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Bootstrap starting.
I20260810 06:54:44.895064 20960 tablet_bootstrap.cc:654] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:44.895160 20959 tablet_bootstrap.cc:492] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: No bootstrap required, opened a new log
I20260810 06:54:44.895394 20959 ts_tablet_manager.cc:1397] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:54:44.896279 20960 log.cc:824] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:44.897212 20961 tablet_bootstrap.cc:654] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:44.898425 20961 log.cc:824] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:44.901120 20961 tablet_bootstrap.cc:492] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: No bootstrap required, opened a new log
I20260810 06:54:44.901347 20961 ts_tablet_manager.cc:1397] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 06:54:44.901467 20960 tablet_bootstrap.cc:492] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: No bootstrap required, opened a new log
I20260810 06:54:44.901722 20960 ts_tablet_manager.cc:1397] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 06:54:44.904469 20959 raft_consensus.cc:348] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.904955 20959 raft_consensus.cc:374] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:44.905076 20959 raft_consensus.cc:729] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea00de5332b42e38cd97d7ed038d042, State: Initialized, Role: FOLLOWER
I20260810 06:54:44.905655 20959 consensus_queue.cc:260] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.906879 20695 heartbeater.cc:503] Master 127.22.11.190:42141 was elected leader, sending a full tablet report...
I20260810 06:54:44.907408 20959 ts_tablet_manager.cc:1428] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260810 06:54:44.909763 20961 raft_consensus.cc:348] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.910065 20961 raft_consensus.cc:374] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:44.910161 20961 raft_consensus.cc:729] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec582ca422f4e66ac787f240c111e57, State: Initialized, Role: FOLLOWER
I20260810 06:54:44.910676 20961 consensus_queue.cc:260] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.911682 20931 heartbeater.cc:503] Master 127.22.11.190:42141 was elected leader, sending a full tablet report...
I20260810 06:54:44.912333 20961 ts_tablet_manager.cc:1428] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 06:54:44.912279 20960 raft_consensus.cc:348] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.912735 20960 raft_consensus.cc:374] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:44.912864 20960 raft_consensus.cc:729] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1191a60f4adb43e8b628408fcff20284, State: Initialized, Role: FOLLOWER
I20260810 06:54:44.913388 20960 consensus_queue.cc:260] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.914887 20960 ts_tablet_manager.cc:1428] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260810 06:54:44.942634 20972 raft_consensus.cc:482] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:54:44.942870 20972 raft_consensus.cc:504] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.944471 20972 leader_election.cc:288] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ea00de5332b42e38cd97d7ed038d042 (127.22.11.130:42049), 8ec582ca422f4e66ac787f240c111e57 (127.22.11.131:36985)
I20260810 06:54:44.951238 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.944823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:42049 (local address 127.22.11.129:57061)
0810 06:54:44.945458 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:44.945476 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:44.945616 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:44.946038 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:44.946042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:44.946071 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:44.946413 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.946426 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.947603 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.947608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.949241 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.949253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.950515 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.950522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.950693 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.950728 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 06:54:44.950828 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 06:54:44.950955 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":333,"threads_started":1}
I20260810 06:54:44.951228 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.944903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:57061 (local address 127.22.11.130:42049)
0810 06:54:44.945084 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.945088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.945561 (+ 473us) server_negotiation.cc:407] Connection header received
0810 06:54:44.945768 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.945773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.945856 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.945950 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:44.946564 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.947484 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.949397 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.950323 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.950430 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.950482 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.950845 (+ 363us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.950967 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:54:44.951670 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.945593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:36985 (local address 127.22.11.129:42675)
0810 06:54:44.946224 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:44.946245 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:44.946350 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:44.946658 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:44.946662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:44.946685 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:44.947060 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.947071 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.948229 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.948233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.949732 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:44.949746 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.950985 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.950991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:44.951171 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.951202 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:54:44.951298 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 06:54:44.951413 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":311,"threads_started":1}
I20260810 06:54:44.951692 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.945699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:42675 (local address 127.22.11.131:36985)
0810 06:54:44.945919 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.945926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.946332 (+ 406us) server_negotiation.cc:407] Connection header received
0810 06:54:44.946465 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.946470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.946557 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.946620 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:44.947191 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.948091 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.949880 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.950781 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.950917 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.950980 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.951345 (+ 365us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.951467 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:54:44.952453 20859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8965bdecf489b82e85103ce43c761" candidate_uuid: "1191a60f4adb43e8b628408fcff20284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ec582ca422f4e66ac787f240c111e57" is_pre_election: true
I20260810 06:54:44.952474 20626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8965bdecf489b82e85103ce43c761" candidate_uuid: "1191a60f4adb43e8b628408fcff20284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea00de5332b42e38cd97d7ed038d042" is_pre_election: true
I20260810 06:54:44.952953 20859 raft_consensus.cc:2454] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1191a60f4adb43e8b628408fcff20284 in term 0.
I20260810 06:54:44.952948 20626 raft_consensus.cc:2454] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1191a60f4adb43e8b628408fcff20284 in term 0.
I20260810 06:54:44.953542 20274 leader_election.cc:302] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1191a60f4adb43e8b628408fcff20284, 4ea00de5332b42e38cd97d7ed038d042; no voters:
I20260810 06:54:44.953886 20972 raft_consensus.cc:2781] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:54:44.954016 20972 raft_consensus.cc:482] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:54:44.954123 20972 raft_consensus.cc:3037] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:44.956470 20972 raft_consensus.cc:504] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.957070 20972 leader_election.cc:288] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [CANDIDATE]: Term 1 election: Requested vote from peers 4ea00de5332b42e38cd97d7ed038d042 (127.22.11.130:42049), 8ec582ca422f4e66ac787f240c111e57 (127.22.11.131:36985)
I20260810 06:54:44.957414 20626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8965bdecf489b82e85103ce43c761" candidate_uuid: "1191a60f4adb43e8b628408fcff20284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ea00de5332b42e38cd97d7ed038d042"
I20260810 06:54:44.957487 20859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50c8965bdecf489b82e85103ce43c761" candidate_uuid: "1191a60f4adb43e8b628408fcff20284" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ec582ca422f4e66ac787f240c111e57"
I20260810 06:54:44.957578 20626 raft_consensus.cc:3037] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:44.957690 20859 raft_consensus.cc:3037] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:44.959710 20626 raft_consensus.cc:2454] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1191a60f4adb43e8b628408fcff20284 in term 1.
I20260810 06:54:44.959833 20859 raft_consensus.cc:2454] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1191a60f4adb43e8b628408fcff20284 in term 1.
I20260810 06:54:44.960088 20274 leader_election.cc:302] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1191a60f4adb43e8b628408fcff20284, 4ea00de5332b42e38cd97d7ed038d042; no voters:
I20260810 06:54:44.960309 20972 raft_consensus.cc:2781] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:44.960724 20972 raft_consensus.cc:686] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 1 LEADER]: Becoming Leader. State: Replica: 1191a60f4adb43e8b628408fcff20284, State: Running, Role: LEADER
I20260810 06:54:44.961169 20972 consensus_queue.cc:237] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } }
I20260810 06:54:44.965166 20133 catalog_manager.cc:5495] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1191a60f4adb43e8b628408fcff20284 (127.22.11.129). New cstate: current_term: 1 leader_uuid: "1191a60f4adb43e8b628408fcff20284" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1191a60f4adb43e8b628408fcff20284" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 38347 } health_report { overall_health: HEALTHY } } }
I20260810 06:54:44.991436 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:44.986064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52188 (local address 127.22.11.129:38347)
0810 06:54:44.986266 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:44.986272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:44.986369 (+ 97us) server_negotiation.cc:407] Connection header received
0810 06:54:44.986577 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:44.986582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:44.986674 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:44.986795 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:54:44.987185 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.987880 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.989415 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:44.989814 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:44.989920 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:44.990374 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:54:44.990841 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:54:44.990971 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:44.991124 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 06:54:44.991201 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:54:44.998982 20859 raft_consensus.cc:1261] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 1 FOLLOWER]: Refusing update from remote peer 1191a60f4adb43e8b628408fcff20284: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:44.999012 20626 raft_consensus.cc:1261] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 1 FOLLOWER]: Refusing update from remote peer 1191a60f4adb43e8b628408fcff20284: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:44.999811 20981 consensus_queue.cc:1035] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ea00de5332b42e38cd97d7ed038d042" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:45.000146 20972 consensus_queue.cc:1035] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ec582ca422f4e66ac787f240c111e57" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 36985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:45.022415 20132 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:45.022760 20132 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:45.026598 20132 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ee3f720b996a47a1a8156e1062a988d4: Sending DeleteTablet for 3 replicas of tablet 50c8965bdecf489b82e85103ce43c761
I20260810 06:54:45.027622 20591 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c8965bdecf489b82e85103ce43c761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:45 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:36998
I20260810 06:54:45.027724 20828 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c8965bdecf489b82e85103ce43c761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:45 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:45850
I20260810 06:54:45.027835 20358 tablet_service.cc:1514] Processing DeleteTablet for tablet 50c8965bdecf489b82e85103ce43c761 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:45 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52180
I20260810 06:54:45.029443 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 20247
I20260810 06:54:45.036419 21010 tablet_replica.cc:330] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: stopping tablet replica
I20260810 06:54:45.036870 21010 raft_consensus.cc:2229] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:45.037386 21010 raft_consensus.cc:2258] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:45.038173 21011 tablet_replica.cc:330] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: stopping tablet replica
I20260810 06:54:45.038524 21011 raft_consensus.cc:2229] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:54:45.039127 21011 raft_consensus.cc:2258] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:45.041128 21010 ts_tablet_manager.cc:1905] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:45.042217 21011 ts_tablet_manager.cc:1905] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:45.047999 21010 ts_tablet_manager.cc:1918] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:45.048238 21010 log.cc:1196] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/50c8965bdecf489b82e85103ce43c761
I20260810 06:54:45.048852 21010 ts_tablet_manager.cc:1939] T 50c8965bdecf489b82e85103ce43c761 P 8ec582ca422f4e66ac787f240c111e57: Deleting consensus metadata
I20260810 06:54:45.049546 21011 ts_tablet_manager.cc:1918] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:45.049782 21011 log.cc:1196] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/50c8965bdecf489b82e85103ce43c761
I20260810 06:54:45.050351 21011 ts_tablet_manager.cc:1939] T 50c8965bdecf489b82e85103ce43c761 P 1191a60f4adb43e8b628408fcff20284: Deleting consensus metadata
I20260810 06:54:45.051108 20106 catalog_manager.cc:4841] TS 8ec582ca422f4e66ac787f240c111e57 (127.22.11.131:36985): tablet 50c8965bdecf489b82e85103ce43c761 (table test-table [id=fbd177f58bbe41198179e0ff62cabb3f]) successfully deleted
I20260810 06:54:45.051966 20106 catalog_manager.cc:4841] TS 1191a60f4adb43e8b628408fcff20284 (127.22.11.129:38347): tablet 50c8965bdecf489b82e85103ce43c761 (table test-table [id=fbd177f58bbe41198179e0ff62cabb3f]) successfully deleted
I20260810 06:54:45.052662 21009 tablet_replica.cc:330] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: stopping tablet replica
I20260810 06:54:45.053071 21009 raft_consensus.cc:2229] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:45.053546 21009 raft_consensus.cc:2258] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:45.055351 21009 ts_tablet_manager.cc:1905] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:54:45 dist-test-slave-f7th krb5kdc[20049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:54:45.062052 21009 ts_tablet_manager.cc:1918] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:45.062216 21009 log.cc:1196] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/50c8965bdecf489b82e85103ce43c761
I20260810 06:54:45.062731 21009 ts_tablet_manager.cc:1939] T 50c8965bdecf489b82e85103ce43c761 P 4ea00de5332b42e38cd97d7ed038d042: Deleting consensus metadata
I20260810 06:54:45.064505 20106 catalog_manager.cc:4841] TS 4ea00de5332b42e38cd97d7ed038d042 (127.22.11.130:42049): tablet 50c8965bdecf489b82e85103ce43c761 (table test-table [id=fbd177f58bbe41198179e0ff62cabb3f]) successfully deleted
I20260810 06:54:45.066290 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:45.031565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52202 (local address 127.22.11.129:38347)
0810 06:54:45.031745 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:45.031752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:45.051978 (+ 20226us) server_negotiation.cc:407] Connection header received
0810 06:54:45.052195 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:45.052200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:45.052290 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:45.052392 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:45.053832 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:45.054798 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:45.056096 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:45.056549 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:45.056685 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:45.060581 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:45.060592 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:45.060605 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:45.060674 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:45.063541 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:45.064207 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:45.064215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:45.064219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:45.064421 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:45.064909 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:45.064913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:45.064915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:45.065262 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:45.065512 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:45.065815 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 06:54:45.065992 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:54:45.066958 20468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:45.169492 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 20247
I20260810 06:54:45.194865 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 20489
Aug 10 06:54:45 dist-test-slave-f7th krb5kdc[20049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:54:45.209298 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:45.195614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37006 (local address 127.22.11.130:42049)
0810 06:54:45.195781 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:45.195787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:45.195809 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:54:45.195981 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:45.195985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:45.196051 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:45.196150 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:45.197377 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:45.198122 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:45.199364 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:45.199754 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:45.199855 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:45.203998 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:45.204014 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:45.204031 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:45.204069 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:45.206791 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:45.207446 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:45.207453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:45.207455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:45.207592 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:45.207924 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:45.207927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:45.207928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:45.208317 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:45.208557 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:45.208805 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 06:54:45.208974 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:54:45.209980 20685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:45.297895 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 20489
I20260810 06:54:45.320062 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 20704
Aug 10 06:54:45 dist-test-slave-f7th krb5kdc[20049](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:54:45.338135 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:45.320792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.22.11.131:36985)
0810 06:54:45.320947 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:45.320951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:45.320970 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:54:45.321138 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:45.321142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:45.321207 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:45.321287 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:45.322669 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:45.323408 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:45.324862 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:45.325275 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:45.325370 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:45.332972 (+ 7602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:45.332989 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:45.333007 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:45.333044 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:45.335647 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:45.336347 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:45.336354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:45.336356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:45.336507 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:45.336844 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:45.336847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:45.336849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:45.337197 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:45.337420 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:45.337713 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:54:45.337879 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:54:45.338821 20917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:45.427914 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 20704
I20260810 06:54:45.450603 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 20076
I20260810 06:54:45.451821 20184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:45.559547 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 20076
2026-08-10T06:54:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4365 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T06:54:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:54:45Z Disabled control of system clock
I20260810 06:54:45.648440 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40301
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:41247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:54:45.971674 21026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:45.971948 21026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:45.972070 21026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:45.979115 21026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:54:45.979215 21026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:45.979279 21026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:45.979357 21026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:54:45.979417 21026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:54:45.988396 21026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:41247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:45.989637 21026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:45.991485 21026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:46.002736 21034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:46.002691 21035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:46.002759 21038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:46.003720 21026 server_base.cc:1034] running on GCE node
I20260810 06:54:46.004501 21026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:46.005872 21026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:46.007118 21026 hybrid_clock.cc:648] HybridClock initialized: now 1786344886007068 us; error 39 us; skew 500 ppm
I20260810 06:54:46.007459 21026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:46.009656 21026 webserver.cc:460] Webserver started at http://127.22.11.190:38001/ using document root <none> and password file <none>
I20260810 06:54:46.010314 21026 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:46.010414 21026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:46.010715 21026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:46.013811 21026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "83a682a990024c9faae2af109661e71c"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "16ed36aeb76972c0b931bae6ea2df485"
server_key_iv: "20719facf6aac0758e56786ffbf62343"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.014587 21026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "83a682a990024c9faae2af109661e71c"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "16ed36aeb76972c0b931bae6ea2df485"
server_key_iv: "20719facf6aac0758e56786ffbf62343"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.019606 21026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260810 06:54:46.022908 21045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:46.024159 21026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:54:46.024322 21026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "83a682a990024c9faae2af109661e71c"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "16ed36aeb76972c0b931bae6ea2df485"
server_key_iv: "20719facf6aac0758e56786ffbf62343"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.024509 21026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:46.043900 21026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:46.044672 21026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:46.044922 21026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:46.061865 21026 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:40301
I20260810 06:54:46.061885 21132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:40301 every 8 connection(s)
I20260810 06:54:46.063086 21026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:54:46.066670 21133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:46.071362 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 21026
I20260810 06:54:46.071560 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:54:46.072126 22574 external_mini_cluster.cc:1428] Setting key 3cc71c849d4358ea931b90ccc007deaf
I20260810 06:54:46.077647 21133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c: Bootstrap starting.
I20260810 06:54:46.081034 21133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:46.082149 21133 log.cc:824] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:46.085255 21133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c: No bootstrap required, opened a new log
I20260810 06:54:46.085404 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:46.073976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40890 (local address 127.22.11.190:40301)
0810 06:54:46.074938 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:46.074951 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:46.074993 (+ 42us) server_negotiation.cc:407] Connection header received
0810 06:54:46.076739 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:46.076762 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:46.077163 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:46.077460 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:46.078043 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:46.079329 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:46.081216 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:46.081875 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:46.082001 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:46.083202 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:46.083212 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:46.083229 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:46.083833 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:46.083905 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:46.083924 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:46.084005 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:46.084219 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:54:46.084497 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":382,"threads_started":1}
I20260810 06:54:46.092394 21133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a682a990024c9faae2af109661e71c" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40301 } }
I20260810 06:54:46.092753 21133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:46.092845 21133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a682a990024c9faae2af109661e71c, State: Initialized, Role: FOLLOWER
I20260810 06:54:46.093392 21133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a682a990024c9faae2af109661e71c" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40301 } }
I20260810 06:54:46.093678 21133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:54:46.093822 21133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:54:46.094053 21133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:46.096107 21133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a682a990024c9faae2af109661e71c" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40301 } }
I20260810 06:54:46.096549 21133 leader_election.cc:302] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83a682a990024c9faae2af109661e71c; no voters:
I20260810 06:54:46.097100 21133 leader_election.cc:288] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:54:46.097210 21141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:46.097859 21141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [term 1 LEADER]: Becoming Leader. State: Replica: 83a682a990024c9faae2af109661e71c, State: Running, Role: LEADER
I20260810 06:54:46.098373 21141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a682a990024c9faae2af109661e71c" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40301 } }
I20260810 06:54:46.098942 21133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:54:46.100939 21144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83a682a990024c9faae2af109661e71c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a682a990024c9faae2af109661e71c" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40301 } } }
I20260810 06:54:46.101346 21144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:46.100984 21145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83a682a990024c9faae2af109661e71c. Latest consensus state: current_term: 1 leader_uuid: "83a682a990024c9faae2af109661e71c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a682a990024c9faae2af109661e71c" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40301 } } }
I20260810 06:54:46.101573 21145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:46.102833 21148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:54:46.107700 21148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:54:46.115820 21148 catalog_manager.cc:1347] Generated new cluster ID: 2c4ec59433ec4e45a7db82a870805eb8
I20260810 06:54:46.115927 21148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:54:46.143203 21148 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:54:46.144441 21148 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:54:46.151926 21148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c: Generated new TSK 0
I20260810 06:54:46.152751 21148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:54:46.163801 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40301
--builtin_ntp_servers=127.22.11.148:41247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:54:46.493667 21177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:46.493914 21177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:46.494015 21177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:46.501106 21177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:46.501195 21177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:46.501329 21177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:54:46.510116 21177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:41247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:46.511245 21177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:46.513006 21177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:46.525173 21191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:46.525239 21188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:46.525269 21189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:46.526587 21177 server_base.cc:1034] running on GCE node
I20260810 06:54:46.527176 21177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:46.528079 21177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:46.529270 21177 hybrid_clock.cc:648] HybridClock initialized: now 1786344886529228 us; error 30 us; skew 500 ppm
I20260810 06:54:46.529577 21177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:46.531611 21177 webserver.cc:460] Webserver started at http://127.22.11.129:39829/ using document root <none> and password file <none>
I20260810 06:54:46.532241 21177 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:46.532341 21177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:46.532681 21177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:46.535373 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "92f834208a0d87d63c6d2adbee5509a1"
server_key_iv: "e588fd65b1b859939bfcdea2fa202012"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.536109 21177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "92f834208a0d87d63c6d2adbee5509a1"
server_key_iv: "e588fd65b1b859939bfcdea2fa202012"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.540792 21177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 06:54:46.543946 21212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:46.545128 21177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:54:46.545269 21177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "92f834208a0d87d63c6d2adbee5509a1"
server_key_iv: "e588fd65b1b859939bfcdea2fa202012"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.545450 21177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:46.569515 21177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:46.570317 21177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:46.570609 21177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:46.571523 21177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:46.572960 21177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:46.573060 21177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:46.573165 21177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:46.573238 21177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:46.603928 21177 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:42751
I20260810 06:54:46.603989 21420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:42751 every 8 connection(s)
I20260810 06:54:46.605288 21177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:54:46.613742 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 21177
I20260810 06:54:46.613897 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:54:46.614351 22574 external_mini_cluster.cc:1428] Setting key b8d21e0aa027adfc164700f1c47f238b
I20260810 06:54:46.617228 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40301
--builtin_ntp_servers=127.22.11.148:41247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:54:46.617441 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:46.607689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:35461 (local address 127.22.11.190:40301)
0810 06:54:46.608090 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:46.608095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:46.609335 (+ 1240us) server_negotiation.cc:407] Connection header received
0810 06:54:46.611090 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:46.611096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:46.611175 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:46.611276 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:46.612313 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:46.613064 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:46.614648 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:46.615089 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:46.615205 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:46.616651 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:46.616658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:46.616663 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:46.616682 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:46.616735 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:46.616752 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:46.616842 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:46.617016 (+ 174us) server_negotiation.cc:299] Negotiation successful
0810 06:54:46.617113 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":205,"threads_started":1}
I20260810 06:54:46.618304 21425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:46.608240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40301 (local address 127.22.11.129:35461)
0810 06:54:46.609187 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:46.609231 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:46.610836 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:46.611500 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:46.611508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:46.611598 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:46.612151 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:46.612168 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:46.613201 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:46.613205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:46.614508 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:46.614521 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:46.615336 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:46.615345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:46.615638 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:46.616319 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:54:46.616322 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:46.616359 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:54:46.616437 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:46.616834 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:46.616840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:46.616846 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 06:54:46.616980 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 06:54:46.617253 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":276,"threads_started":1}
I20260810 06:54:46.619781 21422 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40301
I20260810 06:54:46.620082 21422 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:46.620961 21422 heartbeater.cc:511] Master 127.22.11.190:40301 requested a full tablet report, sending...
I20260810 06:54:46.623008 21074 ts_manager.cc:194] Registered new tserver with Master: 5fdae1bc081c4dcb9bc0efd6151cdc96 (127.22.11.129:42751)
I20260810 06:54:46.624696 21074 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.129:35461
W20260810 06:54:46.947458 21426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:46.947757 21426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:46.947890 21426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:46.954967 21426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:46.955070 21426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:46.955212 21426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:54:46.964118 21426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:41247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:46.965377 21426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:46.967167 21426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:46.979832 21442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:46.979916 21440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:46.979918 21437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:46.980700 21426 server_base.cc:1034] running on GCE node
I20260810 06:54:46.981292 21426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:46.982173 21426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:46.983389 21426 hybrid_clock.cc:648] HybridClock initialized: now 1786344886983327 us; error 44 us; skew 500 ppm
I20260810 06:54:46.983690 21426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:46.985954 21426 webserver.cc:460] Webserver started at http://127.22.11.130:45241/ using document root <none> and password file <none>
I20260810 06:54:46.986608 21426 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:46.986708 21426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:46.987030 21426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:46.989838 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "934f1bcc86c747579846c520bb7a7e6d"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "5660ccea642ef365ba21f98b872d1841"
server_key_iv: "19d267202b2d515fafa2b3651a24059c"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.990625 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "934f1bcc86c747579846c520bb7a7e6d"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "5660ccea642ef365ba21f98b872d1841"
server_key_iv: "19d267202b2d515fafa2b3651a24059c"
server_key_version: "encryptionkey@0"
I20260810 06:54:46.995709 21426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:54:46.999152 21456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:47.000459 21426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:54:47.000649 21426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "934f1bcc86c747579846c520bb7a7e6d"
format_stamp: "Formatted at 2026-08-10 06:54:46 on dist-test-slave-f7th"
server_key: "5660ccea642ef365ba21f98b872d1841"
server_key_iv: "19d267202b2d515fafa2b3651a24059c"
server_key_version: "encryptionkey@0"
I20260810 06:54:47.000854 21426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:47.025183 21426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:47.026019 21426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:47.026309 21426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:47.027288 21426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:47.028853 21426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:47.028952 21426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:47.029057 21426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:47.029129 21426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:47.059396 21426 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:36797
I20260810 06:54:47.059466 21651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:36797 every 8 connection(s)
I20260810 06:54:47.060760 21426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:54:47.066193 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 21426
I20260810 06:54:47.066363 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:54:47.066802 22574 external_mini_cluster.cc:1428] Setting key 7c4ae6c04e04d94f900bd3a1ad07326b
I20260810 06:54:47.069748 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40301
--builtin_ntp_servers=127.22.11.148:41247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:54:47.075362 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.063547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:34145 (local address 127.22.11.190:40301)
0810 06:54:47.063752 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.063758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.065186 (+ 1428us) server_negotiation.cc:407] Connection header received
0810 06:54:47.067670 (+ 2484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.067676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.067760 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.067874 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:47.069102 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.070183 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.072300 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.072937 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.073060 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.074547 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:47.074553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:47.074558 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:47.074578 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:47.074616 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:47.074631 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:47.074715 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.074955 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.075060 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:54:47.076494 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.064000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40301 (local address 127.22.11.130:34145)
0810 06:54:47.065029 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.065077 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:47.067337 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:47.068105 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:47.068113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:47.068205 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:47.068923 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.068951 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.070349 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.070354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.072148 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.072163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.073160 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.073167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.073430 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.074211 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:54:47.074214 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:47.074263 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:54:47.074358 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:47.074725 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:47.074735 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:47.074743 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:54:47.074921 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 06:54:47.075287 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":290,"threads_started":1}
I20260810 06:54:47.077944 21652 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40301
I20260810 06:54:47.078258 21652 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:47.079094 21652 heartbeater.cc:511] Master 127.22.11.190:40301 requested a full tablet report, sending...
I20260810 06:54:47.080510 21074 ts_manager.cc:194] Registered new tserver with Master: 934f1bcc86c747579846c520bb7a7e6d (127.22.11.130:36797)
I20260810 06:54:47.081504 21074 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.130:34145
W20260810 06:54:47.400578 21658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:47.400870 21658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:47.401008 21658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:47.408201 21658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:47.408303 21658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:47.408450 21658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:54:47.417199 21658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:41247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:40301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:47.418416 21658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:47.420521 21658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:47.433357 21664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:47.433451 21672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:47.433389 21666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:47.434350 21658 server_base.cc:1034] running on GCE node
I20260810 06:54:47.434976 21658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:47.435889 21658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:47.437121 21658 hybrid_clock.cc:648] HybridClock initialized: now 1786344887437072 us; error 30 us; skew 500 ppm
I20260810 06:54:47.437435 21658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:47.439563 21658 webserver.cc:460] Webserver started at http://127.22.11.131:41213/ using document root <none> and password file <none>
I20260810 06:54:47.440229 21658 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:47.440330 21658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:47.440649 21658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:47.443428 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "ea0cd119c18146f19e46796dd0fbf7c8"
format_stamp: "Formatted at 2026-08-10 06:54:47 on dist-test-slave-f7th"
server_key: "a919139a87176009a7c79caea65dc110"
server_key_iv: "f4d3731cd166ece1f89fd8c24fd640d2"
server_key_version: "encryptionkey@0"
I20260810 06:54:47.444205 21658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "ea0cd119c18146f19e46796dd0fbf7c8"
format_stamp: "Formatted at 2026-08-10 06:54:47 on dist-test-slave-f7th"
server_key: "a919139a87176009a7c79caea65dc110"
server_key_iv: "f4d3731cd166ece1f89fd8c24fd640d2"
server_key_version: "encryptionkey@0"
I20260810 06:54:47.449314 21658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260810 06:54:47.452739 21681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:47.454109 21658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:54:47.454267 21658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "ea0cd119c18146f19e46796dd0fbf7c8"
format_stamp: "Formatted at 2026-08-10 06:54:47 on dist-test-slave-f7th"
server_key: "a919139a87176009a7c79caea65dc110"
server_key_iv: "f4d3731cd166ece1f89fd8c24fd640d2"
server_key_version: "encryptionkey@0"
I20260810 06:54:47.454461 21658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:47.474823 21658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:47.475572 21658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:47.475857 21658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:47.476768 21658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:47.478145 21658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:47.478251 21658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:47.478353 21658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:47.478421 21658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:47.507678 21658 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:44089
I20260810 06:54:47.507745 21862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:44089 every 8 connection(s)
I20260810 06:54:47.508978 21658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:54:47.509235 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 21658
I20260810 06:54:47.509407 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:54:47.509958 22574 external_mini_cluster.cc:1428] Setting key 833339b0ad3d4a238dedb6848c77eb3a
I20260810 06:54:47.520493 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.511439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:33237 (local address 127.22.11.190:40301)
0810 06:54:47.511638 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.511643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.513303 (+ 1660us) server_negotiation.cc:407] Connection header received
0810 06:54:47.514964 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.514968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.515032 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.515114 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:47.516049 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.516844 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.518267 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.518668 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.518758 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.519890 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:47.519895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:47.519898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:47.519915 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:47.519944 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:47.519955 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:47.520022 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.520177 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.520253 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:54:47.521350 21865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.511974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40301 (local address 127.22.11.131:33237)
0810 06:54:47.513164 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.513207 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:47.514753 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:47.515286 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:47.515293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:47.515372 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:47.515926 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.515941 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.516963 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.516966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.518150 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.518161 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.518827 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.518831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.519019 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.519622 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:54:47.519624 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:47.519658 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:54:47.519731 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:47.520027 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:47.520032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:47.520037 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:54:47.520161 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 06:54:47.520413 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":301,"threads_started":1}
I20260810 06:54:47.522604 21863 heartbeater.cc:347] Connected to a master server at 127.22.11.190:40301
I20260810 06:54:47.522856 21863 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:47.523578 21863 heartbeater.cc:511] Master 127.22.11.190:40301 requested a full tablet report, sending...
I20260810 06:54:47.524961 21072 ts_manager.cc:194] Registered new tserver with Master: ea0cd119c18146f19e46796dd0fbf7c8 (127.22.11.131:44089)
I20260810 06:54:47.525787 21072 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.131:33237
I20260810 06:54:47.537108 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:54:47.546145 21424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.540953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.22.11.190:40301)
0810 06:54:47.541090 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.541094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.541577 (+ 483us) server_negotiation.cc:407] Connection header received
0810 06:54:47.541722 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.541725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.541792 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.541864 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:47.542270 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.542919 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.544183 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.544588 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.544711 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.545633 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:47.545637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:47.545641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:47.545657 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:47.545686 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:47.545697 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:47.545760 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.545844 (+ 84us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.545924 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 06:54:47.549899 21072 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:54:47.556715 21072 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:54:47.588210 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.570526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:36797 (local address 127.0.0.1:52514)
0810 06:54:47.580853 (+ 10327us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.580872 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:47.580992 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:47.581701 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:47.581705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:47.581733 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:47.582610 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.582617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.583877 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.583883 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.585846 (+ 1963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.585858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.587290 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.587297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.587483 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.587535 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 06:54:47.587660 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 06:54:47.587838 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10223,"mutex_wait_us":6,"thread_start_us":10105,"threads_started":1}
I20260810 06:54:47.588227 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.571117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52514 (local address 127.22.11.130:36797)
0810 06:54:47.580520 (+ 9403us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.580529 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.581120 (+ 591us) server_negotiation.cc:407] Connection header received
0810 06:54:47.581229 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.581236 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.581478 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.581871 (+ 393us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:47.582741 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.583682 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.586042 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.587109 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.587239 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.587475 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.587732 (+ 257us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.587901 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9257,"thread_start_us":9156,"threads_started":1}
I20260810 06:54:47.589222 21885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.568276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:44089 (local address 127.0.0.1:40090)
0810 06:54:47.569628 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.569681 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:47.569860 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:47.581738 (+ 11878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:47.581745 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:47.581764 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:47.582263 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.582275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.584089 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.584098 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.586665 (+ 2567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.586681 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.588363 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.588369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.588527 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.588640 (+ 113us) client_negotiation.cc:769] Sending connection context
0810 06:54:47.588720 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 06:54:47.588842 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":194,"thread_start_us":1864,"threads_started":1}
I20260810 06:54:47.589342 21883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.568919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58890)
0810 06:54:47.570411 (+ 1492us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.570427 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:47.570526 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:47.581598 (+ 11072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:47.581606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:47.581635 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:47.582099 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.582117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.584093 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.584098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.586669 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.586681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.588530 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.588536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.588735 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.588767 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:54:47.588825 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 06:54:47.588945 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1364,"mutex_wait_us":152,"spinlock_wait_cycles":3840,"thread_start_us":683,"threads_started":1}
I20260810 06:54:47.589476 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.568587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.22.11.131:44089)
0810 06:54:47.581261 (+ 12674us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.581268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.581295 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:54:47.581377 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.581383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.581560 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.581870 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:47.582440 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.583954 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.586886 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.588146 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.588642 (+ 496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.588913 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.588978 (+ 65us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.589136 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12511,"thread_start_us":12321,"threads_started":1}
I20260810 06:54:47.590811 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.569440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.22.11.129:42751)
0810 06:54:47.581046 (+ 11606us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.581055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.581087 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:54:47.581185 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.581191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.581385 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.581556 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:47.582294 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.583924 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.586938 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.588208 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.589948 (+ 1740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.590163 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.590231 (+ 68us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.590403 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11448,"thread_start_us":11174,"threads_started":1}
I20260810 06:54:47.592715 21531 tablet_service.cc:1467] Processing CreateTablet for tablet bd7243496ffb474ab03fcbdc8bbf9b92 (DEFAULT_TABLE table=test-table [id=1071e20a3ca44c559ab86e42fd17a82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:47.593585 21754 tablet_service.cc:1467] Processing CreateTablet for tablet bd7243496ffb474ab03fcbdc8bbf9b92 (DEFAULT_TABLE table=test-table [id=1071e20a3ca44c559ab86e42fd17a82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:47.594245 21531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd7243496ffb474ab03fcbdc8bbf9b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:47.594051 21298 tablet_service.cc:1467] Processing CreateTablet for tablet bd7243496ffb474ab03fcbdc8bbf9b92 (DEFAULT_TABLE table=test-table [id=1071e20a3ca44c559ab86e42fd17a82d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:47.594794 21754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd7243496ffb474ab03fcbdc8bbf9b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:47.595316 21298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd7243496ffb474ab03fcbdc8bbf9b92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:47.610715 21891 tablet_bootstrap.cc:492] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Bootstrap starting.
I20260810 06:54:47.612408 21895 tablet_bootstrap.cc:492] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: Bootstrap starting.
I20260810 06:54:47.613735 21894 tablet_bootstrap.cc:492] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Bootstrap starting.
I20260810 06:54:47.615366 21895 tablet_bootstrap.cc:654] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:47.616379 21894 tablet_bootstrap.cc:654] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:47.616549 21895 log.cc:824] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:47.617569 21894 log.cc:824] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:47.617895 21891 tablet_bootstrap.cc:654] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:47.619081 21891 log.cc:824] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:47.619761 21895 tablet_bootstrap.cc:492] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: No bootstrap required, opened a new log
I20260810 06:54:47.619998 21895 ts_tablet_manager.cc:1397] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 06:54:47.620271 21894 tablet_bootstrap.cc:492] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: No bootstrap required, opened a new log
I20260810 06:54:47.620543 21894 ts_tablet_manager.cc:1397] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260810 06:54:47.621975 21891 tablet_bootstrap.cc:492] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: No bootstrap required, opened a new log
I20260810 06:54:47.622228 21891 ts_tablet_manager.cc:1397] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260810 06:54:47.627157 21894 raft_consensus.cc:348] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.627521 21894 raft_consensus.cc:374] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:47.627633 21894 raft_consensus.cc:729] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea0cd119c18146f19e46796dd0fbf7c8, State: Initialized, Role: FOLLOWER
I20260810 06:54:47.628481 21894 consensus_queue.cc:260] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.628732 21422 heartbeater.cc:503] Master 127.22.11.190:40301 was elected leader, sending a full tablet report...
I20260810 06:54:47.630125 21895 raft_consensus.cc:348] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.630537 21895 raft_consensus.cc:374] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:47.630628 21895 raft_consensus.cc:729] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fdae1bc081c4dcb9bc0efd6151cdc96, State: Initialized, Role: FOLLOWER
I20260810 06:54:47.631271 21895 consensus_queue.cc:260] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.632651 21891 raft_consensus.cc:348] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.633148 21891 raft_consensus.cc:374] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:47.633282 21891 raft_consensus.cc:729] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 934f1bcc86c747579846c520bb7a7e6d, State: Initialized, Role: FOLLOWER
I20260810 06:54:47.634042 21891 consensus_queue.cc:260] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.635860 21863 heartbeater.cc:503] Master 127.22.11.190:40301 was elected leader, sending a full tablet report...
I20260810 06:54:47.636152 21895 ts_tablet_manager.cc:1428] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260810 06:54:47.636570 21894 ts_tablet_manager.cc:1428] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260810 06:54:47.637471 21652 heartbeater.cc:503] Master 127.22.11.190:40301 was elected leader, sending a full tablet report...
I20260810 06:54:47.638278 21891 ts_tablet_manager.cc:1428] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260810 06:54:47.650312 21902 raft_consensus.cc:482] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:54:47.650542 21902 raft_consensus.cc:504] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.652233 21902 leader_election.cc:288] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea0cd119c18146f19e46796dd0fbf7c8 (127.22.11.131:44089), 934f1bcc86c747579846c520bb7a7e6d (127.22.11.130:36797)
I20260810 06:54:47.659196 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.652609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:48103 (local address 127.22.11.131:44089)
0810 06:54:47.652821 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.652828 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.653293 (+ 465us) server_negotiation.cc:407] Connection header received
0810 06:54:47.653480 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.653485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.653560 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.653646 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:47.654209 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.655141 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.657271 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.658246 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.658359 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.658407 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.658825 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.658953 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:54:47.659409 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.652561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:44089 (local address 127.22.11.129:48103)
0810 06:54:47.653186 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.653203 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:47.653348 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:47.653691 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:47.653696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:47.653730 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:47.654071 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.654083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.655277 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.655283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.657091 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.657106 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.658418 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.658425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.658592 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.658626 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:54:47.658780 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 06:54:47.658983 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":52,"thread_start_us":362,"threads_started":1}
I20260810 06:54:47.659505 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.653395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:51349 (local address 127.22.11.130:36797)
0810 06:54:47.653599 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.653605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.654006 (+ 401us) server_negotiation.cc:407] Connection header received
0810 06:54:47.654154 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.654159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.654241 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.654308 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:47.654952 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.655880 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.657739 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.658611 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.658764 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.658825 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.659150 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.659269 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:54:47.659550 21906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.653291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:36797 (local address 127.22.11.129:51349)
0810 06:54:47.653909 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.653922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:47.654040 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:47.654359 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:47.654364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:47.654389 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:47.654740 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.654752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.656019 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.656023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.657557 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:47.657572 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.658801 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.658808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:47.658997 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.659029 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:54:47.659112 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 06:54:47.659223 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":1,"thread_start_us":315,"threads_started":1}
I20260810 06:54:47.660538 21575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7243496ffb474ab03fcbdc8bbf9b92" candidate_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "934f1bcc86c747579846c520bb7a7e6d" is_pre_election: true
I20260810 06:54:47.660538 21784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7243496ffb474ab03fcbdc8bbf9b92" candidate_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" is_pre_election: true
I20260810 06:54:47.661141 21575 raft_consensus.cc:2454] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fdae1bc081c4dcb9bc0efd6151cdc96 in term 0.
I20260810 06:54:47.661141 21784 raft_consensus.cc:2454] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fdae1bc081c4dcb9bc0efd6151cdc96 in term 0.
I20260810 06:54:47.661791 21213 leader_election.cc:302] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fdae1bc081c4dcb9bc0efd6151cdc96, ea0cd119c18146f19e46796dd0fbf7c8; no voters:
I20260810 06:54:47.662119 21902 raft_consensus.cc:2781] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:54:47.662272 21902 raft_consensus.cc:482] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:54:47.662425 21902 raft_consensus.cc:3037] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:47.664521 21902 raft_consensus.cc:504] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.665190 21902 leader_election.cc:288] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [CANDIDATE]: Term 1 election: Requested vote from peers ea0cd119c18146f19e46796dd0fbf7c8 (127.22.11.131:44089), 934f1bcc86c747579846c520bb7a7e6d (127.22.11.130:36797)
I20260810 06:54:47.665508 21784 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7243496ffb474ab03fcbdc8bbf9b92" candidate_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea0cd119c18146f19e46796dd0fbf7c8"
I20260810 06:54:47.665617 21575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7243496ffb474ab03fcbdc8bbf9b92" candidate_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "934f1bcc86c747579846c520bb7a7e6d"
I20260810 06:54:47.665709 21784 raft_consensus.cc:3037] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:47.665822 21575 raft_consensus.cc:3037] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:47.668081 21784 raft_consensus.cc:2454] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fdae1bc081c4dcb9bc0efd6151cdc96 in term 1.
I20260810 06:54:47.668253 21575 raft_consensus.cc:2454] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fdae1bc081c4dcb9bc0efd6151cdc96 in term 1.
I20260810 06:54:47.668417 21213 leader_election.cc:302] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fdae1bc081c4dcb9bc0efd6151cdc96, ea0cd119c18146f19e46796dd0fbf7c8; no voters:
I20260810 06:54:47.668692 21902 raft_consensus.cc:2781] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:47.669131 21902 raft_consensus.cc:686] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 1 LEADER]: Becoming Leader. State: Replica: 5fdae1bc081c4dcb9bc0efd6151cdc96, State: Running, Role: LEADER
I20260810 06:54:47.669622 21902 consensus_queue.cc:237] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } }
I20260810 06:54:47.673774 21074 catalog_manager.cc:5495] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fdae1bc081c4dcb9bc0efd6151cdc96 (127.22.11.129). New cstate: current_term: 1 leader_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fdae1bc081c4dcb9bc0efd6151cdc96" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 42751 } health_report { overall_health: HEALTHY } } }
I20260810 06:54:47.701872 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.696751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.22.11.129:42751)
0810 06:54:47.696980 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.696986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.697014 (+ 28us) server_negotiation.cc:407] Connection header received
0810 06:54:47.697117 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.697120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.697213 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.697340 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:54:47.697698 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.698383 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.699813 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.700225 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.700331 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.700841 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:54:47.701310 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:54:47.701440 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.701550 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.701628 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:54:47.709916 21784 raft_consensus.cc:1261] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 1 FOLLOWER]: Refusing update from remote peer 5fdae1bc081c4dcb9bc0efd6151cdc96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:47.709921 21575 raft_consensus.cc:1261] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 1 FOLLOWER]: Refusing update from remote peer 5fdae1bc081c4dcb9bc0efd6151cdc96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:47.710762 21907 consensus_queue.cc:1035] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea0cd119c18146f19e46796dd0fbf7c8" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 44089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:47.711124 21902 consensus_queue.cc:1035] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "934f1bcc86c747579846c520bb7a7e6d" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:47.734912 21072 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:47.735203 21072 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:47.738869 21072 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83a682a990024c9faae2af109661e71c: Sending DeleteTablet for 3 replicas of tablet bd7243496ffb474ab03fcbdc8bbf9b92
I20260810 06:54:47.739877 21754 tablet_service.cc:1514] Processing DeleteTablet for tablet bd7243496ffb474ab03fcbdc8bbf9b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:47 UTC) from {username='slave'} at 127.0.0.1:40090
I20260810 06:54:47.740221 21298 tablet_service.cc:1514] Processing DeleteTablet for tablet bd7243496ffb474ab03fcbdc8bbf9b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:47 UTC) from {username='slave'} at 127.0.0.1:58890
I20260810 06:54:47.740478 21531 tablet_service.cc:1514] Processing DeleteTablet for tablet bd7243496ffb474ab03fcbdc8bbf9b92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:47 UTC) from {username='slave'} at 127.0.0.1:52514
I20260810 06:54:47.741272 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 21177
I20260810 06:54:47.751569 21926 tablet_replica.cc:330] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: stopping tablet replica
I20260810 06:54:47.752152 21926 raft_consensus.cc:2229] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:54:47.752961 21926 raft_consensus.cc:2258] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:47.755326 21926 ts_tablet_manager.cc:1905] T bd7243496ffb474ab03fcbdc8bbf9b92 P 5fdae1bc081c4dcb9bc0efd6151cdc96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:47.757128 21927 tablet_replica.cc:330] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: stopping tablet replica
I20260810 06:54:47.757517 21927 raft_consensus.cc:2229] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:47.758898 21925 tablet_replica.cc:330] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: stopping tablet replica
I20260810 06:54:47.759325 21927 raft_consensus.cc:2258] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:47.759276 21925 raft_consensus.cc:2229] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:47.759734 21925 raft_consensus.cc:2258] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:47.760586 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.742484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58912 (local address 127.22.11.129:42751)
0810 06:54:47.742672 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.742679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.748908 (+ 6229us) server_negotiation.cc:407] Connection header received
0810 06:54:47.749843 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.749849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.749941 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.750016 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:47.751199 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.752458 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.756082 (+ 3624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.756801 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.756898 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.758583 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:47.758598 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:47.758614 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:47.759547 (+ 933us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:47.759613 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:47.759638 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:47.759742 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.759951 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.760064 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:54:47.761725 21925 ts_tablet_manager.cc:1905] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:47.761725 21927 ts_tablet_manager.cc:1905] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:47.761792 21411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:47.768720 21927 ts_tablet_manager.cc:1918] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:47.768714 21925 ts_tablet_manager.cc:1918] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:47.768882 21927 log.cc:1196] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/bd7243496ffb474ab03fcbdc8bbf9b92
I20260810 06:54:47.768878 21925 log.cc:1196] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/bd7243496ffb474ab03fcbdc8bbf9b92
I20260810 06:54:47.769407 21927 ts_tablet_manager.cc:1939] T bd7243496ffb474ab03fcbdc8bbf9b92 P 934f1bcc86c747579846c520bb7a7e6d: Deleting consensus metadata
I20260810 06:54:47.769407 21925 ts_tablet_manager.cc:1939] T bd7243496ffb474ab03fcbdc8bbf9b92 P ea0cd119c18146f19e46796dd0fbf7c8: Deleting consensus metadata
I20260810 06:54:47.771229 21048 catalog_manager.cc:4841] TS 934f1bcc86c747579846c520bb7a7e6d (127.22.11.130:36797): tablet bd7243496ffb474ab03fcbdc8bbf9b92 (table test-table [id=1071e20a3ca44c559ab86e42fd17a82d]) successfully deleted
I20260810 06:54:47.771522 21048 catalog_manager.cc:4841] TS ea0cd119c18146f19e46796dd0fbf7c8 (127.22.11.131:44089): tablet bd7243496ffb474ab03fcbdc8bbf9b92 (table test-table [id=1071e20a3ca44c559ab86e42fd17a82d]) successfully deleted
I20260810 06:54:47.856554 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 21177
I20260810 06:54:47.882972 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 21426
W20260810 06:54:47.882970 21049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.11.129:42751 (error 108)
I20260810 06:54:47.884824 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.884351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58928)
0810 06:54:47.884529 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.884678 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 06:54:47.885075 21049 catalog_manager.cc:4584] TS 5fdae1bc081c4dcb9bc0efd6151cdc96 (127.22.11.129:42751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd7243496ffb474ab03fcbdc8bbf9b92: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
I20260810 06:54:47.889655 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.883790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52530 (local address 127.22.11.130:36797)
0810 06:54:47.883951 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:47.883957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:47.884046 (+ 89us) server_negotiation.cc:407] Connection header received
0810 06:54:47.884244 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:47.884248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:47.884351 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:47.884453 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:47.884943 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.885728 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.887043 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:47.887430 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:47.887514 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:47.888410 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:47.888417 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:47.888430 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:47.889033 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:47.889078 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:47.889098 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:47.889183 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:47.889276 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 06:54:47.889358 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:54:47.890302 21645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:47.938773 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.938214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58940)
0810 06:54:47.938457 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.938573 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 06:54:47.939775 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.939404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58956)
0810 06:54:47.939540 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.939624 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:54:47.977010 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 21426
I20260810 06:54:48.000161 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:47.999666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58962)
0810 06:54:47.999881 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:47.999971 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:54:48.001051 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 21658
I20260810 06:54:48.001361 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:48.000996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58966)
0810 06:54:48.001153 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:48.001238 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:54:48.007758 21884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:48.001796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.22.11.131:44089)
0810 06:54:48.001965 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:48.001971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:48.001991 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:54:48.002172 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:48.002177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:48.002266 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:48.002351 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:48.002793 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:48.003584 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:48.004949 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:48.005341 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:48.005453 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:48.006423 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:48.006433 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:48.006449 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:48.007052 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:48.007101 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:48.007121 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:48.007222 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:48.007351 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 06:54:48.007466 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 06:54:48.008358 21856 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:48.082149 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:48.081580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58968)
0810 06:54:48.081804 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:48.081938 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 06:54:48.083191 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:48.082766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:42751 (local address 127.0.0.1:58972)
0810 06:54:48.082973 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:48.083070 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:42751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 06:54:48.095074 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 21658
I20260810 06:54:48.119374 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 21026
I20260810 06:54:48.120394 21128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:48.227916 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 21026
2026-08-10T06:54:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2671 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:54:48 dist-test-slave-f7th krb5kdc[21943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:54:48 dist-test-slave-f7th krb5kdc[21943](info): set up 2 sockets
Aug 10 06:54:48 dist-test-slave-f7th krb5kdc[21943](info): commencing operation
krb5kdc: starting...
W20260810 06:54:49.495015 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.008s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:54:49 dist-test-slave-f7th krb5kdc[21943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:54:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:54:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:54:49.769307 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37941
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:37941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:50.096240 21968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:50.096509 21968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:50.096653 21968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:50.103806 21968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:54:50.103911 21968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:50.103995 21968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:50.104065 21968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:54:50.104123 21968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:54:50.113822 21968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:37941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:50.115070 21968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:50.116359 21968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:50.127472 21979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:50.127422 21980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:50.127518 21984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:50.128480 21968 server_base.cc:1034] running on GCE node
I20260810 06:54:50.129212 21968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:50.130559 21968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:50.131747 21968 hybrid_clock.cc:648] HybridClock initialized: now 1786344890131693 us; error 45 us; skew 500 ppm
I20260810 06:54:50.132061 21968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:50 dist-test-slave-f7th krb5kdc[21943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344890, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:50.135111 21968 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:54:50.136456 21968 webserver.cc:460] Webserver started at http://127.22.11.190:33095/ using document root <none> and password file <none>
I20260810 06:54:50.137130 21968 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:50.137248 21968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:50.137547 21968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:50.140372 21968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "19bb46b55ed04c078f368e5c66109b05"
format_stamp: "Formatted at 2026-08-10 06:54:50 on dist-test-slave-f7th"
server_key: "2daf63c81ab6fb12182893f69fca2d54"
server_key_iv: "b291e7f1b0d946d7e79c611439d00e98"
server_key_version: "encryptionkey@0"
I20260810 06:54:50.141125 21968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "19bb46b55ed04c078f368e5c66109b05"
format_stamp: "Formatted at 2026-08-10 06:54:50 on dist-test-slave-f7th"
server_key: "2daf63c81ab6fb12182893f69fca2d54"
server_key_iv: "b291e7f1b0d946d7e79c611439d00e98"
server_key_version: "encryptionkey@0"
I20260810 06:54:50.146096 21968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:54:50.149408 21993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:50.150664 21968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:54:50.150830 21968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "19bb46b55ed04c078f368e5c66109b05"
format_stamp: "Formatted at 2026-08-10 06:54:50 on dist-test-slave-f7th"
server_key: "2daf63c81ab6fb12182893f69fca2d54"
server_key_iv: "b291e7f1b0d946d7e79c611439d00e98"
server_key_version: "encryptionkey@0"
I20260810 06:54:50.150998 21968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:50.168771 21968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:50.172721 21968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:50.172986 21968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:50.189973 21968 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:37941
I20260810 06:54:50.189991 22078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:37941 every 8 connection(s)
I20260810 06:54:50.191269 21968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:54:50.193462 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 21968
I20260810 06:54:50.193648 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:54:50.194103 22574 external_mini_cluster.cc:1428] Setting key 078549e2309cd1383202b9dcb5e0077e
I20260810 06:54:50.194979 22080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:50.206516 22080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05: Bootstrap starting.
Aug 10 06:54:50 dist-test-slave-f7th krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:50.209653 22080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:50.210937 22080 log.cc:824] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:50.214125 22080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05: No bootstrap required, opened a new log
I20260810 06:54:50.216106 22085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:50.195818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.22.11.190:37941)
0810 06:54:50.196562 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:50.196573 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:50.196631 (+ 58us) server_negotiation.cc:407] Connection header received
0810 06:54:50.198235 (+ 1604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:50.198254 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:50.198610 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:50.198999 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:50.200449 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:50.201874 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:50.203192 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:50.203873 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:50.204053 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:50.208194 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:50.208202 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:50.208214 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:50.208249 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:50.211728 (+ 3479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:50.212519 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:50.212526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:50.212534 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:50.212786 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:50.213140 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:50.213145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:50.213148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:50.213806 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:50.214085 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:50.214421 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 06:54:50.214855 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":334,"threads_started":1}
I20260810 06:54:50.222496 22080 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb46b55ed04c078f368e5c66109b05" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37941 } }
I20260810 06:54:50.222807 22080 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:50.222878 22080 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19bb46b55ed04c078f368e5c66109b05, State: Initialized, Role: FOLLOWER
I20260810 06:54:50.223341 22080 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb46b55ed04c078f368e5c66109b05" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37941 } }
I20260810 06:54:50.223529 22080 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:54:50.223634 22080 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:54:50.223757 22080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:50.225939 22080 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb46b55ed04c078f368e5c66109b05" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37941 } }
I20260810 06:54:50.226367 22080 leader_election.cc:302] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19bb46b55ed04c078f368e5c66109b05; no voters:
I20260810 06:54:50.226886 22080 leader_election.cc:288] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:54:50.227021 22088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:50.227646 22088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [term 1 LEADER]: Becoming Leader. State: Replica: 19bb46b55ed04c078f368e5c66109b05, State: Running, Role: LEADER
I20260810 06:54:50.228130 22088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb46b55ed04c078f368e5c66109b05" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37941 } }
I20260810 06:54:50.228792 22080 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:54:50.231868 22089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19bb46b55ed04c078f368e5c66109b05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb46b55ed04c078f368e5c66109b05" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37941 } } }
I20260810 06:54:50.232268 22089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:50.232978 22091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19bb46b55ed04c078f368e5c66109b05. Latest consensus state: current_term: 1 leader_uuid: "19bb46b55ed04c078f368e5c66109b05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb46b55ed04c078f368e5c66109b05" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 37941 } } }
I20260810 06:54:50.233186 22104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:54:50.233309 22091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05 [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:50.237550 22104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:54:50.245510 22104 catalog_manager.cc:1347] Generated new cluster ID: 6e6c8131955c444aafd3a9aaee061f75
I20260810 06:54:50.245612 22104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:54:50.254135 22104 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:54:50.255868 22104 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:54:50.265223 22104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05: Generated new TSK 0
I20260810 06:54:50.266132 22104 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:54:50.333844 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37941
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:50.673839 22121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:50.674113 22121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:50.674252 22121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:50.684789 22121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:50.684912 22121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:50.685099 22121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:54:50.694501 22121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:50.695623 22121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:50.696863 22121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:50.709296 22128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:50.709240 22129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:50.709307 22134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:50.710054 22121 server_base.cc:1034] running on GCE node
I20260810 06:54:50.710597 22121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:50.711459 22121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:50.712688 22121 hybrid_clock.cc:648] HybridClock initialized: now 1786344890712631 us; error 44 us; skew 500 ppm
I20260810 06:54:50.712981 22121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:50 dist-test-slave-f7th krb5kdc[21943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344890, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:50.716140 22121 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:54:50.717553 22121 webserver.cc:460] Webserver started at http://127.22.11.129:45085/ using document root <none> and password file <none>
I20260810 06:54:50.718171 22121 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:50.718286 22121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:50.718575 22121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:50.721351 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "c8bfc61d487d4b65a37346febf953a00"
format_stamp: "Formatted at 2026-08-10 06:54:50 on dist-test-slave-f7th"
server_key: "66eed216643ca22228d8bda7bdbf4487"
server_key_iv: "d39ba2c7d8aaa5da0afb294c9a425274"
server_key_version: "encryptionkey@0"
I20260810 06:54:50.722097 22121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "c8bfc61d487d4b65a37346febf953a00"
format_stamp: "Formatted at 2026-08-10 06:54:50 on dist-test-slave-f7th"
server_key: "66eed216643ca22228d8bda7bdbf4487"
server_key_iv: "d39ba2c7d8aaa5da0afb294c9a425274"
server_key_version: "encryptionkey@0"
I20260810 06:54:50.726943 22121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260810 06:54:50.730204 22142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:50.731501 22121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:54:50.731645 22121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "c8bfc61d487d4b65a37346febf953a00"
format_stamp: "Formatted at 2026-08-10 06:54:50 on dist-test-slave-f7th"
server_key: "66eed216643ca22228d8bda7bdbf4487"
server_key_iv: "d39ba2c7d8aaa5da0afb294c9a425274"
server_key_version: "encryptionkey@0"
I20260810 06:54:50.731822 22121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:50.759502 22121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:50.763096 22121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:50.763350 22121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:50.764289 22121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:50.765743 22121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:50.765815 22121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:50.765904 22121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:50.765946 22121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:50.795454 22121 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:35295
I20260810 06:54:50.795502 22337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:35295 every 8 connection(s)
I20260810 06:54:50.796727 22121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:54:50.804971 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 22121
I20260810 06:54:50.805140 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:54:50.805567 22574 external_mini_cluster.cc:1428] Setting key 4cc4f83c4e16880802f2978d97956ead
Aug 10 06:54:50 dist-test-slave-f7th krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344890, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:50.815922 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:50.799079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:38189 (local address 127.22.11.190:37941)
0810 06:54:50.799485 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:50.799490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:50.800403 (+ 913us) server_negotiation.cc:407] Connection header received
0810 06:54:50.802239 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:50.802246 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:50.802325 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:50.802423 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:50.803865 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:50.804584 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:50.806216 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:50.806686 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:50.806809 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:50.810378 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:50.810385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:50.810390 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:50.810427 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:50.813198 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:50.813834 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:50.813838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:50.813841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:50.813978 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:50.814340 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:50.814344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:50.814347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:50.814658 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:50.814788 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:50.815370 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 06:54:50.815589 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":207,"threads_started":1}
I20260810 06:54:50.816602 22344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:50.799542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:37941 (local address 127.22.11.129:38189)
0810 06:54:50.800249 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:50.800294 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:50.801970 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:50.802656 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:50.802664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:50.803170 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:50.803702 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:50.803719 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:50.804747 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:50.804751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:50.806066 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:50.806083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:50.806887 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:50.806893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:50.807085 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:50.807765 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:50.807795 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:50.810172 (+ 2377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:50.813313 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:50.813319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:50.813330 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:50.813727 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:50.814067 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:50.814070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:50.814072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:50.814248 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:50.814760 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:50.814765 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:50.815040 (+ 275us) client_negotiation.cc:769] Sending connection context
0810 06:54:50.815246 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 06:54:50.815605 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":287,"threads_started":1}
I20260810 06:54:50.817929 22339 heartbeater.cc:347] Connected to a master server at 127.22.11.190:37941
I20260810 06:54:50.818234 22339 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:50.819018 22339 heartbeater.cc:511] Master 127.22.11.190:37941 requested a full tablet report, sending...
I20260810 06:54:50.821070 22014 ts_manager.cc:194] Registered new tserver with Master: c8bfc61d487d4b65a37346febf953a00 (127.22.11.129:35295)
I20260810 06:54:50.822826 22014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:38189
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:54:50.868755 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37941
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:51.188369 22349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:51.188686 22349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:51.188820 22349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:51.195937 22349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:51.196062 22349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:51.196214 22349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:54:51.205679 22349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:51.206811 22349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:51.208106 22349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:51.220518 22362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:51.220569 22359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:51.220599 22358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:51.225847 22349 server_base.cc:1034] running on GCE node
I20260810 06:54:51.226384 22349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:51.227247 22349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:51.228435 22349 hybrid_clock.cc:648] HybridClock initialized: now 1786344891228375 us; error 56 us; skew 500 ppm
I20260810 06:54:51.228758 22349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:51 dist-test-slave-f7th krb5kdc[21943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:51.231765 22349 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:54:51.233299 22349 webserver.cc:460] Webserver started at http://127.22.11.130:43297/ using document root <none> and password file <none>
I20260810 06:54:51.233963 22349 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:51.234084 22349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:51.234372 22349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:51.237172 22349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "707e60622c334efe9bf90c933c2222f8"
format_stamp: "Formatted at 2026-08-10 06:54:51 on dist-test-slave-f7th"
server_key: "007af60f6bfcb10c9cb6a48b71f486f8"
server_key_iv: "00306e02c1da650ebf61612c798995f4"
server_key_version: "encryptionkey@0"
I20260810 06:54:51.237890 22349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "707e60622c334efe9bf90c933c2222f8"
format_stamp: "Formatted at 2026-08-10 06:54:51 on dist-test-slave-f7th"
server_key: "007af60f6bfcb10c9cb6a48b71f486f8"
server_key_iv: "00306e02c1da650ebf61612c798995f4"
server_key_version: "encryptionkey@0"
I20260810 06:54:51.242781 22349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 06:54:51.245939 22374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:51.247318 22349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 06:54:51.247462 22349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "707e60622c334efe9bf90c933c2222f8"
format_stamp: "Formatted at 2026-08-10 06:54:51 on dist-test-slave-f7th"
server_key: "007af60f6bfcb10c9cb6a48b71f486f8"
server_key_iv: "00306e02c1da650ebf61612c798995f4"
server_key_version: "encryptionkey@0"
I20260810 06:54:51.247658 22349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:51.276116 22349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:51.279801 22349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:51.280058 22349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:51.280957 22349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:51.282243 22349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:51.282313 22349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:51.282425 22349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:51.282474 22349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:51.311450 22349 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:42793
I20260810 06:54:51.311491 22589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:42793 every 8 connection(s)
I20260810 06:54:51.312688 22349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:54:51.313416 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 22349
I20260810 06:54:51.313621 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:54:51.314234 22574 external_mini_cluster.cc:1428] Setting key 2a50dc2541d69b26b69c8ea15bdeacd2
Aug 10 06:54:51 dist-test-slave-f7th krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:51.332111 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.315556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:34861 (local address 127.22.11.190:37941)
0810 06:54:51.315740 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:51.315747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:51.316983 (+ 1236us) server_negotiation.cc:407] Connection header received
0810 06:54:51.318840 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:51.318845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:51.318929 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:51.319015 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:51.320455 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.321230 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.322725 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.323156 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.323268 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.326746 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:51.326751 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:51.326754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:51.326785 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:51.329242 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:51.329940 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:51.329945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:51.329948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:51.330106 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:51.330465 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:51.330470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:51.330473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:51.330796 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:51.330936 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:51.331544 (+ 608us) server_negotiation.cc:299] Negotiation successful
0810 06:54:51.331767 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:54:51.332849 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.316070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:37941 (local address 127.22.11.130:34861)
0810 06:54:51.316835 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:51.316873 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:51.318592 (+ 1719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:51.319250 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:51.319258 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:51.319753 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:51.320295 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.320313 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.321377 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.321381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.322586 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.322599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.323371 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.323379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.323589 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.324193 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:51.324213 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:51.326542 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:51.329373 (+ 2831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:51.329379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:51.329390 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:51.329837 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:51.330199 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:51.330202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:51.330204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:51.330368 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:51.330894 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:51.330899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:51.331200 (+ 301us) client_negotiation.cc:769] Sending connection context
0810 06:54:51.331416 (+ 216us) client_negotiation.cc:240] Negotiation successful
0810 06:54:51.331778 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":284,"threads_started":1}
I20260810 06:54:51.334127 22590 heartbeater.cc:347] Connected to a master server at 127.22.11.190:37941
I20260810 06:54:51.334390 22590 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:51.335098 22590 heartbeater.cc:511] Master 127.22.11.190:37941 requested a full tablet report, sending...
I20260810 06:54:51.336547 22014 ts_manager.cc:194] Registered new tserver with Master: 707e60622c334efe9bf90c933c2222f8 (127.22.11.130:42793)
I20260810 06:54:51.337327 22014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:34861
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:54:51.375417 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37941
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:37693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:51.697566 22595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:51.697836 22595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:51.697978 22595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:51.705061 22595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:51.705170 22595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:51.705346 22595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:54:51.714551 22595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:37693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:37941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:51.715631 22595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:51.716881 22595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:51.729354 22612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:51.729399 22607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:51.729594 22606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:51.730125 22595 server_base.cc:1034] running on GCE node
I20260810 06:54:51.730695 22595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:51.731624 22595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:51.732820 22595 hybrid_clock.cc:648] HybridClock initialized: now 1786344891732763 us; error 53 us; skew 500 ppm
I20260810 06:54:51.733111 22595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:51 dist-test-slave-f7th krb5kdc[21943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:51.736215 22595 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:54:51.737679 22595 webserver.cc:460] Webserver started at http://127.22.11.131:34933/ using document root <none> and password file <none>
I20260810 06:54:51.738309 22595 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:51.738408 22595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:51.738667 22595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:51.741358 22595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "ef760df371c144f68b7000629da76aab"
format_stamp: "Formatted at 2026-08-10 06:54:51 on dist-test-slave-f7th"
server_key: "027008201f4a94faeac85462e0bc350f"
server_key_iv: "1d49e0e7a1fcbbc779cc13db2563c1b2"
server_key_version: "encryptionkey@0"
I20260810 06:54:51.742045 22595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "ef760df371c144f68b7000629da76aab"
format_stamp: "Formatted at 2026-08-10 06:54:51 on dist-test-slave-f7th"
server_key: "027008201f4a94faeac85462e0bc350f"
server_key_iv: "1d49e0e7a1fcbbc779cc13db2563c1b2"
server_key_version: "encryptionkey@0"
I20260810 06:54:51.746754 22595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:54:51.749861 22624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:51.751011 22595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:54:51.751147 22595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "ef760df371c144f68b7000629da76aab"
format_stamp: "Formatted at 2026-08-10 06:54:51 on dist-test-slave-f7th"
server_key: "027008201f4a94faeac85462e0bc350f"
server_key_iv: "1d49e0e7a1fcbbc779cc13db2563c1b2"
server_key_version: "encryptionkey@0"
I20260810 06:54:51.751343 22595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:51.784870 22595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:51.788940 22595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:51.789197 22595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:51.790105 22595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:51.791363 22595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:51.791435 22595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:51.791556 22595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:51.791606 22595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:51.821103 22595 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:35545
I20260810 06:54:51.821166 22814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:35545 every 8 connection(s)
I20260810 06:54:51.822270 22595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:54:51.825976 22339 heartbeater.cc:503] Master 127.22.11.190:37941 was elected leader, sending a full tablet report...
I20260810 06:54:51.829533 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 22595
I20260810 06:54:51.829727 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:54:51.830183 22574 external_mini_cluster.cc:1428] Setting key 285a220a3560bed0c0e27e48ca961f25
Aug 10 06:54:51 dist-test-slave-f7th krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344891, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:51.841228 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.824539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:47643 (local address 127.22.11.190:37941)
0810 06:54:51.824718 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:51.824722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:51.825841 (+ 1119us) server_negotiation.cc:407] Connection header received
0810 06:54:51.827691 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:51.827694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:51.827784 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:51.827861 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:51.829334 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.830199 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.832075 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.832520 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.832645 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.835950 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:51.835955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:51.835958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:51.835991 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:51.838690 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:51.839288 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:51.839292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:51.839294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:51.839419 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:51.839798 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:51.839803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:51.839805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:51.840099 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:51.840223 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:51.840806 (+ 583us) server_negotiation.cc:299] Negotiation successful
0810 06:54:51.840967 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:54:51.842031 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.825009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:37941 (local address 127.22.11.131:47643)
0810 06:54:51.825666 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:51.825703 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:51.827436 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:51.828060 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:51.828067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:51.828573 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:51.829178 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.829205 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.830379 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.830384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.831941 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.831953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.832774 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.832783 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.833021 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.833645 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:51.833689 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:51.835741 (+ 2052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:51.838800 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:51.838806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:51.838816 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:51.839200 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:51.839495 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:51.839497 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:51.839499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:51.839661 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:51.840202 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:51.840207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:51.840475 (+ 268us) client_negotiation.cc:769] Sending connection context
0810 06:54:51.840712 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 06:54:51.841057 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":265,"threads_started":1}
I20260810 06:54:51.843185 22817 heartbeater.cc:347] Connected to a master server at 127.22.11.190:37941
I20260810 06:54:51.843449 22817 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:51.844208 22817 heartbeater.cc:511] Master 127.22.11.190:37941 requested a full tablet report, sending...
I20260810 06:54:51.845618 22014 ts_manager.cc:194] Registered new tserver with Master: ef760df371c144f68b7000629da76aab (127.22.11.131:35545)
I20260810 06:54:51.846371 22014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:47643
I20260810 06:54:51.856848 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:54:51.873862 22342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.860820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.22.11.190:37941)
0810 06:54:51.860983 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:51.860989 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:51.861398 (+ 409us) server_negotiation.cc:407] Connection header received
0810 06:54:51.861549 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:51.861554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:51.861617 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:51.861693 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:51.863080 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.863756 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.865080 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.865476 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.865581 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.868315 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:51.868321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:51.868326 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:51.868359 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:51.870835 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:51.871553 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:51.871557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:51.871559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:51.871689 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:51.872336 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:51.872340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:51.872342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:51.872730 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:51.872989 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:51.873360 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 06:54:51.873583 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:54:51.877648 22014 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:54:51.884246 22014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:54:51.918994 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.895353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:42793 (local address 127.0.0.1:52544)
0810 06:54:51.910209 (+ 14856us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:51.910227 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:51.910341 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:51.911779 (+ 1438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:51.911786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:51.911816 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:51.912236 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.912251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.913991 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.913997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.916108 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.916123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.917963 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.917970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.918159 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.918215 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 06:54:51.918343 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 06:54:51.918530 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14734,"mutex_wait_us":2,"thread_start_us":15805,"threads_started":1}
I20260810 06:54:51.919078 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.895951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52544 (local address 127.22.11.130:42793)
0810 06:54:51.910941 (+ 14990us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:51.910950 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:51.910984 (+ 34us) server_negotiation.cc:407] Connection header received
0810 06:54:51.911105 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:51.911113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:51.911377 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:51.911584 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:51.912484 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.913859 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.916283 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.917745 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.917880 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.918073 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:51.918553 (+ 480us) server_negotiation.cc:299] Negotiation successful
0810 06:54:51.918727 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14840,"thread_start_us":14719,"threads_started":1}
I20260810 06:54:51.919368 22844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.895295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:35295 (local address 127.0.0.1:41886)
0810 06:54:51.904893 (+ 9598us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:51.904943 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:51.905103 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:51.912669 (+ 7566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:51.912677 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:51.912702 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:51.913117 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.913129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.914957 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.914961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.916508 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.916522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.918589 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.918596 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.918755 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.918782 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 06:54:51.918879 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 06:54:51.919014 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9365,"mutex_wait_us":1,"thread_start_us":15105,"threads_started":1}
I20260810 06:54:51.919996 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.896520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50722 (local address 127.22.11.131:35545)
0810 06:54:51.910808 (+ 14288us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:51.910818 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:51.911687 (+ 869us) server_negotiation.cc:407] Connection header received
0810 06:54:51.911785 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:51.911792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:51.912014 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:51.912211 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:51.912944 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.914501 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.916868 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.918090 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.918218 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.918431 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:51.919483 (+ 1052us) server_negotiation.cc:299] Negotiation successful
0810 06:54:51.919660 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14089,"thread_start_us":14294,"threads_started":1}
I20260810 06:54:51.920145 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.895488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41886 (local address 127.22.11.129:35295)
0810 06:54:51.911821 (+ 16333us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:51.911827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:51.911848 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:54:51.911919 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:51.911924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:51.912087 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:51.912250 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:51.913469 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.914831 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.916736 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.918112 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.918234 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.918449 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:51.919678 (+ 1229us) server_negotiation.cc:299] Negotiation successful
0810 06:54:51.919838 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16172,"thread_start_us":16006,"threads_started":1}
I20260810 06:54:51.920198 22840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:51.895830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:35545 (local address 127.0.0.1:50722)
0810 06:54:51.911431 (+ 15601us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:51.911448 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:51.911566 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:51.912223 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:51.912227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:51.912252 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:51.912697 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.912709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.914636 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.914641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.916681 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:51.916696 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:51.918306 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:51.918312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:51.918474 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:51.918521 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 06:54:51.918606 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:54:51.918731 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15468,"mutex_wait_us":5,"thread_start_us":8688,"threads_started":1}
I20260810 06:54:51.923033 22465 tablet_service.cc:1467] Processing CreateTablet for tablet 358966647cf9433fac1c156dc0077b13 (DEFAULT_TABLE table=test-table [id=0b80ba6ac9024b4eb7d0d46519c39384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:51.923720 22708 tablet_service.cc:1467] Processing CreateTablet for tablet 358966647cf9433fac1c156dc0077b13 (DEFAULT_TABLE table=test-table [id=0b80ba6ac9024b4eb7d0d46519c39384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:51.923702 22225 tablet_service.cc:1467] Processing CreateTablet for tablet 358966647cf9433fac1c156dc0077b13 (DEFAULT_TABLE table=test-table [id=0b80ba6ac9024b4eb7d0d46519c39384]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:51.924453 22465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 358966647cf9433fac1c156dc0077b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:51.925128 22225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 358966647cf9433fac1c156dc0077b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:51.925125 22708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 358966647cf9433fac1c156dc0077b13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:51.936441 22853 tablet_bootstrap.cc:492] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Bootstrap starting.
I20260810 06:54:51.939059 22856 tablet_bootstrap.cc:492] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Bootstrap starting.
I20260810 06:54:51.939939 22853 tablet_bootstrap.cc:654] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:51.940487 22854 tablet_bootstrap.cc:492] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Bootstrap starting.
I20260810 06:54:51.941396 22853 log.cc:824] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:51.942319 22856 tablet_bootstrap.cc:654] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:51.943406 22854 tablet_bootstrap.cc:654] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:51.943420 22856 log.cc:824] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:51.944483 22854 log.cc:824] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:51.945286 22853 tablet_bootstrap.cc:492] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: No bootstrap required, opened a new log
I20260810 06:54:51.945536 22853 ts_tablet_manager.cc:1397] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 06:54:51.947154 22856 tablet_bootstrap.cc:492] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: No bootstrap required, opened a new log
I20260810 06:54:51.947389 22856 ts_tablet_manager.cc:1397] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 06:54:51.947440 22854 tablet_bootstrap.cc:492] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: No bootstrap required, opened a new log
I20260810 06:54:51.947685 22854 ts_tablet_manager.cc:1397] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 06:54:51.954703 22856 raft_consensus.cc:348] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:51.955169 22856 raft_consensus.cc:374] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:51.955315 22856 raft_consensus.cc:729] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef760df371c144f68b7000629da76aab, State: Initialized, Role: FOLLOWER
I20260810 06:54:51.955921 22856 consensus_queue.cc:260] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:51.955852 22853 raft_consensus.cc:348] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:51.956264 22853 raft_consensus.cc:374] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:51.956375 22853 raft_consensus.cc:729] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707e60622c334efe9bf90c933c2222f8, State: Initialized, Role: FOLLOWER
I20260810 06:54:51.957131 22817 heartbeater.cc:503] Master 127.22.11.190:37941 was elected leader, sending a full tablet report...
I20260810 06:54:51.957073 22853 consensus_queue.cc:260] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:51.957777 22856 ts_tablet_manager.cc:1428] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260810 06:54:51.958323 22854 raft_consensus.cc:348] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:51.958750 22854 raft_consensus.cc:374] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:51.958859 22854 raft_consensus.cc:729] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8bfc61d487d4b65a37346febf953a00, State: Initialized, Role: FOLLOWER
I20260810 06:54:51.959450 22854 consensus_queue.cc:260] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:51.961230 22854 ts_tablet_manager.cc:1428] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260810 06:54:51.964131 22590 heartbeater.cc:503] Master 127.22.11.190:37941 was elected leader, sending a full tablet report...
I20260810 06:54:51.965040 22853 ts_tablet_manager.cc:1428] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Time spent starting tablet: real 0.019s user 0.019s sys 0.000s
W20260810 06:54:52.049062 22340 tablet.cc:2369] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:54:52.064672 22591 tablet.cc:2369] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 06:54:52.073799 22819 tablet.cc:2369] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:54:52.122467 22862 raft_consensus.cc:482] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:54:52.122740 22862 raft_consensus.cc:504] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:52.124444 22862 leader_election.cc:288] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8bfc61d487d4b65a37346febf953a00 (127.22.11.129:35295), 707e60622c334efe9bf90c933c2222f8 (127.22.11.130:42793)
I20260810 06:54:52.130520 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.124800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:51971 (local address 127.22.11.129:35295)
0810 06:54:52.124988 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:52.124994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:52.125015 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:54:52.125215 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:52.125220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:52.125303 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:52.125391 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:52.125836 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.126621 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.128490 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.129736 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.129849 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.129903 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:52.130216 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 06:54:52.130338 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:54:52.130622 22821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.124658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:35295 (local address 127.22.11.131:51971)
0810 06:54:52.124886 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:52.124908 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:52.125064 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:52.125432 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:52.125438 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:52.125471 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:52.125653 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:52.125664 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.126756 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.126762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:52.128322 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:52.128334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.129871 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.129877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:52.130069 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.130105 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 06:54:52.130228 (+ 123us) client_negotiation.cc:240] Negotiation successful
0810 06:54:52.130361 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:54:52.131592 22870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.124824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:42793 (local address 127.22.11.131:37901)
0810 06:54:52.125651 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:52.125666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:52.125785 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:52.126074 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:52.126078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:52.126101 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:52.126384 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:52.126392 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.127715 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.127719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:52.129486 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:52.129501 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.131009 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.131015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:52.131178 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.131212 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 06:54:52.131283 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 06:54:52.131362 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":495,"threads_started":1}
I20260810 06:54:52.131700 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.125102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:37901 (local address 127.22.11.130:42793)
0810 06:54:52.125337 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:52.125343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:52.125751 (+ 408us) server_negotiation.cc:407] Connection header received
0810 06:54:52.125911 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:52.125915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:52.125970 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:52.126045 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:52.126535 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.127561 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.129627 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.130687 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.130826 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.130932 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:52.131335 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 06:54:52.131454 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 06:54:52.131714 22260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358966647cf9433fac1c156dc0077b13" candidate_uuid: "ef760df371c144f68b7000629da76aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8bfc61d487d4b65a37346febf953a00" is_pre_election: true
I20260810 06:54:52.132339 22260 raft_consensus.cc:2454] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef760df371c144f68b7000629da76aab in term 0.
I20260810 06:54:52.132597 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358966647cf9433fac1c156dc0077b13" candidate_uuid: "ef760df371c144f68b7000629da76aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707e60622c334efe9bf90c933c2222f8" is_pre_election: true
I20260810 06:54:52.132983 22628 leader_election.cc:302] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8bfc61d487d4b65a37346febf953a00, ef760df371c144f68b7000629da76aab; no voters:
I20260810 06:54:52.133059 22501 raft_consensus.cc:2454] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef760df371c144f68b7000629da76aab in term 0.
I20260810 06:54:52.133332 22862 raft_consensus.cc:2781] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:54:52.133460 22862 raft_consensus.cc:482] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:54:52.133579 22862 raft_consensus.cc:3037] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:52.136044 22862 raft_consensus.cc:504] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:52.136655 22862 leader_election.cc:288] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [CANDIDATE]: Term 1 election: Requested vote from peers c8bfc61d487d4b65a37346febf953a00 (127.22.11.129:35295), 707e60622c334efe9bf90c933c2222f8 (127.22.11.130:42793)
I20260810 06:54:52.137039 22260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358966647cf9433fac1c156dc0077b13" candidate_uuid: "ef760df371c144f68b7000629da76aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8bfc61d487d4b65a37346febf953a00"
I20260810 06:54:52.137127 22501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "358966647cf9433fac1c156dc0077b13" candidate_uuid: "ef760df371c144f68b7000629da76aab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "707e60622c334efe9bf90c933c2222f8"
I20260810 06:54:52.137250 22260 raft_consensus.cc:3037] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:52.137311 22501 raft_consensus.cc:3037] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:52.140363 22501 raft_consensus.cc:2454] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef760df371c144f68b7000629da76aab in term 1.
I20260810 06:54:52.140363 22260 raft_consensus.cc:2454] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef760df371c144f68b7000629da76aab in term 1.
I20260810 06:54:52.140776 22625 leader_election.cc:302] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 707e60622c334efe9bf90c933c2222f8, ef760df371c144f68b7000629da76aab; no voters:
I20260810 06:54:52.141069 22862 raft_consensus.cc:2781] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:52.141485 22862 raft_consensus.cc:686] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 1 LEADER]: Becoming Leader. State: Replica: ef760df371c144f68b7000629da76aab, State: Running, Role: LEADER
I20260810 06:54:52.141938 22862 consensus_queue.cc:237] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } }
I20260810 06:54:52.146129 22014 catalog_manager.cc:5495] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab reported cstate change: term changed from 0 to 1, leader changed from <none> to ef760df371c144f68b7000629da76aab (127.22.11.131). New cstate: current_term: 1 leader_uuid: "ef760df371c144f68b7000629da76aab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef760df371c144f68b7000629da76aab" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 35545 } health_report { overall_health: HEALTHY } } }
I20260810 06:54:52.208297 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.203304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50728 (local address 127.22.11.131:35545)
0810 06:54:52.203518 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:52.203524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:52.203636 (+ 112us) server_negotiation.cc:407] Connection header received
0810 06:54:52.203855 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:52.203859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:52.203940 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:52.204076 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:54:52.204388 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.205104 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.206405 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.206785 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.206884 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.207303 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:54:52.207782 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:54:52.207907 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:52.207986 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 06:54:52.208026 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:54:52.216243 22260 raft_consensus.cc:1261] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 1 FOLLOWER]: Refusing update from remote peer ef760df371c144f68b7000629da76aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:52.216248 22501 raft_consensus.cc:1261] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 1 FOLLOWER]: Refusing update from remote peer ef760df371c144f68b7000629da76aab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:52.217156 22873 consensus_queue.cc:1035] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "707e60622c334efe9bf90c933c2222f8" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:52.217496 22862 consensus_queue.cc:1035] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8bfc61d487d4b65a37346febf953a00" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 35295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:52.228410 22884 mvcc.cc:205] Tried to move back new op lower bound from 7316868678507618304 to 7316868678216835072. Current Snapshot: MvccSnapshot[applied={T|T < 7316868678507618304}]
I20260810 06:54:52.231317 22887 mvcc.cc:205] Tried to move back new op lower bound from 7316868678507618304 to 7316868678216835072. Current Snapshot: MvccSnapshot[applied={T|T < 7316868678507618304}]
I20260810 06:54:52.240831 22013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:52.241097 22013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55288:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:52.244669 22013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19bb46b55ed04c078f368e5c66109b05: Sending DeleteTablet for 3 replicas of tablet 358966647cf9433fac1c156dc0077b13
I20260810 06:54:52.245807 22225 tablet_service.cc:1514] Processing DeleteTablet for tablet 358966647cf9433fac1c156dc0077b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:52 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:41886
I20260810 06:54:52.245828 22465 tablet_service.cc:1514] Processing DeleteTablet for tablet 358966647cf9433fac1c156dc0077b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:52 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:52544
I20260810 06:54:52.246474 22708 tablet_service.cc:1514] Processing DeleteTablet for tablet 358966647cf9433fac1c156dc0077b13 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:52 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:50722
I20260810 06:54:52.247063 22904 tablet_replica.cc:330] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: stopping tablet replica
I20260810 06:54:52.247448 22904 raft_consensus.cc:2229] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:52.247895 22904 raft_consensus.cc:2258] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:52.248590 22902 tablet_replica.cc:330] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: stopping tablet replica
I20260810 06:54:52.248977 22902 raft_consensus.cc:2229] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:52.249429 22902 raft_consensus.cc:2258] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:52.251075 22904 ts_tablet_manager.cc:1905] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:52.251351 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 22121
I20260810 06:54:52.251505 22902 ts_tablet_manager.cc:1905] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:52.258129 22904 ts_tablet_manager.cc:1918] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:52.258304 22904 log.cc:1196] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/358966647cf9433fac1c156dc0077b13
I20260810 06:54:52.258837 22904 ts_tablet_manager.cc:1939] T 358966647cf9433fac1c156dc0077b13 P 707e60622c334efe9bf90c933c2222f8: Deleting consensus metadata
I20260810 06:54:52.258834 22902 ts_tablet_manager.cc:1918] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:52.259133 22902 log.cc:1196] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/358966647cf9433fac1c156dc0077b13
I20260810 06:54:52.259694 22902 ts_tablet_manager.cc:1939] T 358966647cf9433fac1c156dc0077b13 P c8bfc61d487d4b65a37346febf953a00: Deleting consensus metadata
I20260810 06:54:52.260596 21994 catalog_manager.cc:4841] TS 707e60622c334efe9bf90c933c2222f8 (127.22.11.130:42793): tablet 358966647cf9433fac1c156dc0077b13 (table test-table [id=0b80ba6ac9024b4eb7d0d46519c39384]) successfully deleted
I20260810 06:54:52.261411 21997 catalog_manager.cc:4841] TS c8bfc61d487d4b65a37346febf953a00 (127.22.11.129:35295): tablet 358966647cf9433fac1c156dc0077b13 (table test-table [id=0b80ba6ac9024b4eb7d0d46519c39384]) successfully deleted
I20260810 06:54:52.262778 22906 tablet_replica.cc:330] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: stopping tablet replica
I20260810 06:54:52.263181 22906 raft_consensus.cc:2229] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:54:52.263806 22906 raft_consensus.cc:2258] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:52.265940 22906 ts_tablet_manager.cc:1905] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 06:54:52 dist-test-slave-f7th krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:54:52.273082 22906 ts_tablet_manager.cc:1918] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:52.273252 22906 log.cc:1196] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/358966647cf9433fac1c156dc0077b13
I20260810 06:54:52.273852 22906 ts_tablet_manager.cc:1939] T 358966647cf9433fac1c156dc0077b13 P ef760df371c144f68b7000629da76aab: Deleting consensus metadata
I20260810 06:54:52.275612 21995 catalog_manager.cc:4841] TS ef760df371c144f68b7000629da76aab (127.22.11.131:35545): tablet 358966647cf9433fac1c156dc0077b13 (table test-table [id=0b80ba6ac9024b4eb7d0d46519c39384]) successfully deleted
I20260810 06:54:52.276362 22842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.252406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.22.11.129:35295)
0810 06:54:52.252575 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:52.252581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:52.262114 (+ 9533us) server_negotiation.cc:407] Connection header received
0810 06:54:52.262285 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:52.262290 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:52.262392 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:52.262485 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:52.263831 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.264737 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.265971 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.266429 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.266537 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.270777 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:52.270788 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:52.270801 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:52.270833 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:52.273580 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:52.274277 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:52.274284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:52.274286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:52.274422 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:52.274789 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:52.274793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:52.274795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:52.275147 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:52.275628 (+ 481us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:52.275859 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 06:54:52.276066 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:54:52.277016 22329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:52.387392 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 22121
I20260810 06:54:52.415632 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 22349
Aug 10 06:54:52 dist-test-slave-f7th krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:54:52.431581 22847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.416396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52560 (local address 127.22.11.130:42793)
0810 06:54:52.416606 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:52.416637 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:52.416701 (+ 64us) server_negotiation.cc:407] Connection header received
0810 06:54:52.416912 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:52.416916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:52.416991 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:52.417097 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:52.418440 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.419168 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.420787 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.421290 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.421399 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.425859 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:52.425874 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:52.425889 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:52.425922 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:52.428697 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:52.429443 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:52.429450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:52.429452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:52.429586 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:52.429976 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:52.429981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:52.429984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:52.430421 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:52.430726 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:52.431019 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 06:54:52.431231 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 06:54:52.432227 22572 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:52.545755 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 22349
I20260810 06:54:52.570763 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 22595
Aug 10 06:54:52 dist-test-slave-f7th krb5kdc[21943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:54:52.586465 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:52.571554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50734 (local address 127.22.11.131:35545)
0810 06:54:52.571741 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:52.571746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:52.571833 (+ 87us) server_negotiation.cc:407] Connection header received
0810 06:54:52.572033 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:52.572037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:52.572101 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:52.572186 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:52.573567 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.574383 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.575671 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:52.576142 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:52.576254 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:52.580550 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:52.580563 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:52.580583 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:52.580647 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:52.583661 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:52.584328 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:52.584336 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:52.584339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:52.584512 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:52.584902 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:52.584906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:52.584908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:52.585274 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:52.585607 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:52.585877 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 06:54:52.586091 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:54:52.587162 22805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:52.709180 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 22595
I20260810 06:54:52.738245 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 21968
I20260810 06:54:52.739348 22066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:52.847771 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 21968
2026-08-10T06:54:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4617 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T06:54:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:54:52Z Disabled control of system clock
I20260810 06:54:52.938031 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34609
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:54:53.264317 22918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:53.264572 22918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:53.264699 22918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:53.271636 22918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:54:53.271725 22918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:53.271770 22918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:53.271822 22918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:54:53.271867 22918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:54:53.280522 22918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34609
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:53.281701 22918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:53.283388 22918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:53.294271 22927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:53.294310 22931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:53.294314 22933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:53.295218 22918 server_base.cc:1034] running on GCE node
I20260810 06:54:53.295888 22918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:53.297190 22918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:53.298386 22918 hybrid_clock.cc:648] HybridClock initialized: now 1786344893298343 us; error 38 us; skew 500 ppm
I20260810 06:54:53.298722 22918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:53.300843 22918 webserver.cc:460] Webserver started at http://127.22.11.190:33839/ using document root <none> and password file <none>
I20260810 06:54:53.301468 22918 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:53.301576 22918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:53.301889 22918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:53.304822 22918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "776d0f61db0442b0b4d9904a8564ac4d"
format_stamp: "Formatted at 2026-08-10 06:54:53 on dist-test-slave-f7th"
server_key: "d5c347ce480edf9c3dcd24e9b5edd453"
server_key_iv: "37621e001453b6ea215db72a82e8ec0b"
server_key_version: "encryptionkey@0"
I20260810 06:54:53.305619 22918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "776d0f61db0442b0b4d9904a8564ac4d"
format_stamp: "Formatted at 2026-08-10 06:54:53 on dist-test-slave-f7th"
server_key: "d5c347ce480edf9c3dcd24e9b5edd453"
server_key_iv: "37621e001453b6ea215db72a82e8ec0b"
server_key_version: "encryptionkey@0"
I20260810 06:54:53.310438 22918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:54:53.313661 22942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:53.314932 22918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:54:53.315084 22918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "776d0f61db0442b0b4d9904a8564ac4d"
format_stamp: "Formatted at 2026-08-10 06:54:53 on dist-test-slave-f7th"
server_key: "d5c347ce480edf9c3dcd24e9b5edd453"
server_key_iv: "37621e001453b6ea215db72a82e8ec0b"
server_key_version: "encryptionkey@0"
I20260810 06:54:53.315277 22918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:53.334034 22918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:53.334753 22918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:53.335005 22918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:53.352573 22918 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34609
I20260810 06:54:53.352598 23026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34609 every 8 connection(s)
I20260810 06:54:53.353863 22918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:54:53.357347 23028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:53.360778 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 22918
I20260810 06:54:53.360967 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:54:53.361510 22574 external_mini_cluster.cc:1428] Setting key ffe96de46224f5b617e70ec39fc7fe79
I20260810 06:54:53.369637 23028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d: Bootstrap starting.
I20260810 06:54:53.373041 23028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:53.374231 23028 log.cc:824] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:53.375013 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:53.363326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.22.11.190:34609)
0810 06:54:53.364264 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:53.364274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:53.364314 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:54:53.365938 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:53.365959 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:53.366407 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:53.366722 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:53.367332 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:53.368655 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:53.370431 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:53.371106 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:53.371240 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:53.372399 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:53.372406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:53.372420 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:53.372980 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:53.373072 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:53.373094 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:53.373201 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:53.373484 (+ 283us) server_negotiation.cc:299] Negotiation successful
0810 06:54:53.373776 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":353,"threads_started":1}
I20260810 06:54:53.376997 23028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d: No bootstrap required, opened a new log
I20260810 06:54:53.383750 23028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "776d0f61db0442b0b4d9904a8564ac4d" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34609 } }
I20260810 06:54:53.384066 23028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:53.384158 23028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 776d0f61db0442b0b4d9904a8564ac4d, State: Initialized, Role: FOLLOWER
I20260810 06:54:53.384748 23028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "776d0f61db0442b0b4d9904a8564ac4d" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34609 } }
I20260810 06:54:53.384965 23028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:54:53.385093 23028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:54:53.385231 23028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:53.387213 23028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "776d0f61db0442b0b4d9904a8564ac4d" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34609 } }
I20260810 06:54:53.387686 23028 leader_election.cc:302] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 776d0f61db0442b0b4d9904a8564ac4d; no voters:
I20260810 06:54:53.388201 23028 leader_election.cc:288] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:54:53.388325 23033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:53.389015 23033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [term 1 LEADER]: Becoming Leader. State: Replica: 776d0f61db0442b0b4d9904a8564ac4d, State: Running, Role: LEADER
I20260810 06:54:53.389509 23033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "776d0f61db0442b0b4d9904a8564ac4d" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34609 } }
I20260810 06:54:53.390044 23028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:54:53.392328 23034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "776d0f61db0442b0b4d9904a8564ac4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "776d0f61db0442b0b4d9904a8564ac4d" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34609 } } }
I20260810 06:54:53.392814 23034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:53.392903 23036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 776d0f61db0442b0b4d9904a8564ac4d. Latest consensus state: current_term: 1 leader_uuid: "776d0f61db0442b0b4d9904a8564ac4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "776d0f61db0442b0b4d9904a8564ac4d" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34609 } } }
I20260810 06:54:53.393217 23036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:53.394780 23044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:54:53.399648 23044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:54:53.408442 23044 catalog_manager.cc:1347] Generated new cluster ID: 6f01308cc28242b6983a74bb02105bbb
I20260810 06:54:53.408550 23044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:54:53.422909 23044 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:54:53.424552 23044 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:54:53.433174 23044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d: Generated new TSK 0
I20260810 06:54:53.433894 23044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:54:53.445317 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34609
--builtin_ntp_servers=127.22.11.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:54:53.793352 23067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:53.793618 23067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:53.793708 23067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:53.800899 23067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:53.801005 23067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:53.801127 23067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:54:53.810068 23067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:53.811272 23067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:53.813019 23067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:53.825459 23078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:53.825484 23079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:53.825652 23081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:53.826327 23067 server_base.cc:1034] running on GCE node
I20260810 06:54:53.826972 23067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:53.827852 23067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:53.829043 23067 hybrid_clock.cc:648] HybridClock initialized: now 1786344893828992 us; error 43 us; skew 500 ppm
I20260810 06:54:53.829360 23067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:53.831446 23067 webserver.cc:460] Webserver started at http://127.22.11.129:37395/ using document root <none> and password file <none>
I20260810 06:54:53.832064 23067 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:53.832149 23067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:53.832401 23067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:53.835130 23067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "c5399f15cb3d45d1acbbeb936a9a0606"
format_stamp: "Formatted at 2026-08-10 06:54:53 on dist-test-slave-f7th"
server_key: "556265d34adbe06024898598b6c80187"
server_key_iv: "9c13dc4764c5a2e8a55b910a1428ba24"
server_key_version: "encryptionkey@0"
I20260810 06:54:53.835908 23067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "c5399f15cb3d45d1acbbeb936a9a0606"
format_stamp: "Formatted at 2026-08-10 06:54:53 on dist-test-slave-f7th"
server_key: "556265d34adbe06024898598b6c80187"
server_key_iv: "9c13dc4764c5a2e8a55b910a1428ba24"
server_key_version: "encryptionkey@0"
I20260810 06:54:53.840865 23067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:54:53.844141 23096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:53.845352 23067 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:54:53.845510 23067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "c5399f15cb3d45d1acbbeb936a9a0606"
format_stamp: "Formatted at 2026-08-10 06:54:53 on dist-test-slave-f7th"
server_key: "556265d34adbe06024898598b6c80187"
server_key_iv: "9c13dc4764c5a2e8a55b910a1428ba24"
server_key_version: "encryptionkey@0"
I20260810 06:54:53.845695 23067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:53.866776 23067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:53.867528 23067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:53.867805 23067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:53.868716 23067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:53.870074 23067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:53.870163 23067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:53.870267 23067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:53.870349 23067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:53.899734 23067 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:36807
I20260810 06:54:53.899807 23272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:36807 every 8 connection(s)
I20260810 06:54:53.901057 23067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:54:53.910755 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23067
I20260810 06:54:53.910902 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:54:53.911473 22574 external_mini_cluster.cc:1428] Setting key 7f484ff960f1ca4a0ea3afb29ce22bad
I20260810 06:54:53.913166 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:53.903470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:48413 (local address 127.22.11.190:34609)
0810 06:54:53.903923 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:53.903928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:53.905131 (+ 1203us) server_negotiation.cc:407] Connection header received
0810 06:54:53.907035 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:53.907040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:53.907122 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:53.907219 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:53.908237 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:53.908997 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:53.910472 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:53.910896 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:53.911015 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:53.912400 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:53.912405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:53.912409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:53.912426 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:53.912469 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:53.912480 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:53.912554 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:53.912803 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 06:54:53.912878 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":207,"threads_started":1}
I20260810 06:54:53.914350 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:53.904088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34609 (local address 127.22.11.129:48413)
0810 06:54:53.904984 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:53.905024 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:53.906764 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:53.907453 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:53.907461 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:53.907551 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:53.908076 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:53.908092 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:53.909152 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:53.909156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:53.910333 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:53.910345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:53.911119 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:53.911128 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:53.911424 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:53.912105 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:54:53.912108 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:53.912146 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:54:53.912223 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:53.912581 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:53.912594 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:53.912601 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:54:53.912802 (+ 201us) client_negotiation.cc:240] Negotiation successful
0810 06:54:53.913163 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":265,"threads_started":1}
I20260810 06:54:53.914664 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34609
--builtin_ntp_servers=127.22.11.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:54:53.915913 23273 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34609
I20260810 06:54:53.916195 23273 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:53.916901 23273 heartbeater.cc:511] Master 127.22.11.190:34609 requested a full tablet report, sending...
I20260810 06:54:53.918967 22965 ts_manager.cc:194] Registered new tserver with Master: c5399f15cb3d45d1acbbeb936a9a0606 (127.22.11.129:36807)
I20260810 06:54:53.920825 22965 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.129:48413
W20260810 06:54:54.239372 23278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:54.239634 23278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:54.239763 23278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:54.246867 23278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:54.246958 23278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:54.247100 23278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:54:54.255722 23278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:54.256883 23278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:54.258598 23278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:54.270535 23290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:54.270615 23287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:54.270815 23286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:54.271384 23278 server_base.cc:1034] running on GCE node
I20260810 06:54:54.271930 23278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:54.272779 23278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:54.273964 23278 hybrid_clock.cc:648] HybridClock initialized: now 1786344894273913 us; error 46 us; skew 500 ppm
I20260810 06:54:54.274243 23278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:54.276297 23278 webserver.cc:460] Webserver started at http://127.22.11.130:42727/ using document root <none> and password file <none>
I20260810 06:54:54.276973 23278 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:54.277064 23278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:54.277356 23278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:54.280128 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "fe3a3dae31284cd28f5fdcb9e6f97794"
format_stamp: "Formatted at 2026-08-10 06:54:54 on dist-test-slave-f7th"
server_key: "bc39dbe4c6ebb768453088f366e96a70"
server_key_iv: "708fd83acb1bc7930ee8ccfa7aeaaa81"
server_key_version: "encryptionkey@0"
I20260810 06:54:54.280910 23278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "fe3a3dae31284cd28f5fdcb9e6f97794"
format_stamp: "Formatted at 2026-08-10 06:54:54 on dist-test-slave-f7th"
server_key: "bc39dbe4c6ebb768453088f366e96a70"
server_key_iv: "708fd83acb1bc7930ee8ccfa7aeaaa81"
server_key_version: "encryptionkey@0"
I20260810 06:54:54.285773 23278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:54:54.289011 23300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:54.290210 23278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260810 06:54:54.290356 23278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "fe3a3dae31284cd28f5fdcb9e6f97794"
format_stamp: "Formatted at 2026-08-10 06:54:54 on dist-test-slave-f7th"
server_key: "bc39dbe4c6ebb768453088f366e96a70"
server_key_iv: "708fd83acb1bc7930ee8ccfa7aeaaa81"
server_key_version: "encryptionkey@0"
I20260810 06:54:54.290546 23278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:54.315830 23278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:54.316567 23278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:54.316847 23278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:54.317739 23278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:54.319242 23278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:54.319330 23278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:54.319412 23278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:54.319481 23278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:54.349529 23278 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:34773
I20260810 06:54:54.349624 23475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:34773 every 8 connection(s)
I20260810 06:54:54.350755 23278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:54:54.353533 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23278
I20260810 06:54:54.353710 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:54:54.354277 22574 external_mini_cluster.cc:1428] Setting key 9613f1ceecc19d426f1aa2d94cc3405a
I20260810 06:54:54.357471 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34609
--builtin_ntp_servers=127.22.11.148:34159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:54:54.366974 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.353271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:45641 (local address 127.22.11.190:34609)
0810 06:54:54.360054 (+ 6783us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.360061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.360107 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:54:54.360194 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.360205 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.360316 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.360657 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:54.361985 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.362710 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.364218 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.364643 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.364763 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.366159 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:54.366164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:54.366167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:54.366186 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:54.366216 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:54.366227 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:54.366304 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.366527 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.366628 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6594}
I20260810 06:54:54.367971 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.353741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34609 (local address 127.22.11.130:45641)
0810 06:54:54.354760 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:54.355073 (+ 313us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:54.356979 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:54.361156 (+ 4177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:54.361165 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:54.361251 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:54.361814 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.361832 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.362858 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.362862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.364074 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.364087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.364931 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.364945 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.365194 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.365828 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:54:54.365831 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:54.365908 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:54:54.365981 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:54.366317 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:54.366328 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:54.366336 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:54:54.366502 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 06:54:54.366886 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":304,"threads_started":1}
I20260810 06:54:54.369417 23476 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34609
I20260810 06:54:54.369709 23476 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:54.370416 23476 heartbeater.cc:511] Master 127.22.11.190:34609 requested a full tablet report, sending...
I20260810 06:54:54.371889 22965 ts_manager.cc:194] Registered new tserver with Master: fe3a3dae31284cd28f5fdcb9e6f97794 (127.22.11.130:34773)
I20260810 06:54:54.372715 22965 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.130:45641
W20260810 06:54:54.694008 23480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:54.694300 23480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:54.694406 23480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:54.701462 23480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:54.701550 23480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:54.701659 23480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:54:54.710362 23480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:34159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:34609
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:54.711504 23480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:54.713195 23480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:54.725508 23492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:54.725477 23491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:54.725540 23494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:54.726498 23480 server_base.cc:1034] running on GCE node
I20260810 06:54:54.727085 23480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:54.727918 23480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:54.729107 23480 hybrid_clock.cc:648] HybridClock initialized: now 1786344894729051 us; error 47 us; skew 500 ppm
I20260810 06:54:54.729388 23480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:54:54.731431 23480 webserver.cc:460] Webserver started at http://127.22.11.131:34587/ using document root <none> and password file <none>
I20260810 06:54:54.732056 23480 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:54.732158 23480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:54.732445 23480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:54.735172 23480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "c5be907b79ae4f7993809fc436df7350"
format_stamp: "Formatted at 2026-08-10 06:54:54 on dist-test-slave-f7th"
server_key: "d30c739085121810ad210d35bd24f9be"
server_key_iv: "18dd35334da25b93650eed31e58c0b77"
server_key_version: "encryptionkey@0"
I20260810 06:54:54.735930 23480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "c5be907b79ae4f7993809fc436df7350"
format_stamp: "Formatted at 2026-08-10 06:54:54 on dist-test-slave-f7th"
server_key: "d30c739085121810ad210d35bd24f9be"
server_key_iv: "18dd35334da25b93650eed31e58c0b77"
server_key_version: "encryptionkey@0"
I20260810 06:54:54.740653 23480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260810 06:54:54.743736 23501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:54.744958 23480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260810 06:54:54.745105 23480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "c5be907b79ae4f7993809fc436df7350"
format_stamp: "Formatted at 2026-08-10 06:54:54 on dist-test-slave-f7th"
server_key: "d30c739085121810ad210d35bd24f9be"
server_key_iv: "18dd35334da25b93650eed31e58c0b77"
server_key_version: "encryptionkey@0"
I20260810 06:54:54.745285 23480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:54.754998 23480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:54.755682 23480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:54.755937 23480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:54.756850 23480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:54.758131 23480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:54.758222 23480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:54.758327 23480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:54.758397 23480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:54.787561 23480 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:46763
I20260810 06:54:54.787617 23682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:46763 every 8 connection(s)
I20260810 06:54:54.788862 23480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:54:54.794057 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23480
I20260810 06:54:54.794231 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:54:54.794672 22574 external_mini_cluster.cc:1428] Setting key f92659baaf38323a870b271f970ed394
I20260810 06:54:54.800801 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.791360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:51753 (local address 127.22.11.190:34609)
0810 06:54:54.791522 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.791526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.792930 (+ 1404us) server_negotiation.cc:407] Connection header received
0810 06:54:54.794767 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.794773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.794856 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.794945 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:54.796028 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.796870 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.798343 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.798741 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.798846 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.800021 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:54.800025 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:54.800029 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:54.800052 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:54.800082 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:54.800094 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:54.800165 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.800364 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.800468 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:54:54.801677 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.791871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:34609 (local address 127.22.11.131:51753)
0810 06:54:54.792775 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:54.792818 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:54.794486 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:54.795207 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:54.795214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:54.795300 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:54.795838 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.795877 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.797011 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.797015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.798211 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.798224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.798950 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.798959 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.799165 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.799733 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:54:54.799736 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:54.799777 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:54:54.799849 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:54.800179 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:54.800190 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:54.800197 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 06:54:54.800339 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 06:54:54.800733 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":258,"threads_started":1}
I20260810 06:54:54.802821 23684 heartbeater.cc:347] Connected to a master server at 127.22.11.190:34609
I20260810 06:54:54.803078 23684 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:54.803763 23684 heartbeater.cc:511] Master 127.22.11.190:34609 requested a full tablet report, sending...
I20260810 06:54:54.805230 22965 ts_manager.cc:194] Registered new tserver with Master: c5be907b79ae4f7993809fc436df7350 (127.22.11.131:46763)
I20260810 06:54:54.805976 22965 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.131:51753
I20260810 06:54:54.809340 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:54:54.818627 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.813372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.22.11.190:34609)
0810 06:54:54.813560 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.813567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.813941 (+ 374us) server_negotiation.cc:407] Connection header received
0810 06:54:54.814077 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.814081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.814136 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.814201 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:54.814633 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.815285 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.816527 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.816953 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.817048 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.817996 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:54.818003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:54.818007 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:54.818027 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:54.818066 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:54.818079 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:54.818190 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.818308 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.818357 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:54:54.822032 22965 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54738:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:54:54.828547 22965 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:54:54.860898 23711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.839837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:44998)
0810 06:54:54.853336 (+ 13499us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:54.853355 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:54.853663 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:54.854508 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:54.854513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:54.854539 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:54.854975 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.854989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.856466 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.856471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.858660 (+ 2189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.858672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.859943 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.859949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.860157 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.860210 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 06:54:54.860340 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 06:54:54.860516 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13355,"mutex_wait_us":163,"thread_start_us":13506,"threads_started":1}
I20260810 06:54:54.860944 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.841257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.22.11.129:36807)
0810 06:54:54.849031 (+ 7774us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.849039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.853626 (+ 4587us) server_negotiation.cc:407] Connection header received
0810 06:54:54.853862 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.853870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.854119 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.854313 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:54.855246 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.856300 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.858848 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.859771 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.859913 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.860162 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.860394 (+ 232us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.860583 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7606,"thread_start_us":7474,"threads_started":1}
I20260810 06:54:54.862275 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.839594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:34773 (local address 127.0.0.1:32956)
0810 06:54:54.840454 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:54.840491 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:54.840789 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:54.854402 (+ 13613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:54.854406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:54.854426 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:54.854960 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.854978 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.857025 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.857031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.859625 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.859641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.861527 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.861534 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.861703 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.861735 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:54:54.861834 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 06:54:54.861953 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":125,"thread_start_us":4159,"threads_started":1}
I20260810 06:54:54.862383 23709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.839760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32956 (local address 127.22.11.130:34773)
0810 06:54:54.853940 (+ 14180us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.853945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.853967 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:54:54.854049 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.854054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.854212 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.855087 (+ 875us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:54.855335 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.856850 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.859878 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.861319 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.861486 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.861762 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.861870 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.862021 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14031,"thread_start_us":13914,"threads_started":1}
I20260810 06:54:54.863083 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.840223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:46763 (local address 127.0.0.1:45418)
0810 06:54:54.853589 (+ 13366us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:54.853606 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:54.853710 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:54.854343 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:54.854351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:54.854382 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:54.855230 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.855241 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.857104 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.857109 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.859626 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.859641 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.862375 (+ 2734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.862381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.862556 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.862587 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:54:54.862668 (+ 81us) client_negotiation.cc:240] Negotiation successful
0810 06:54:54.862781 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13244,"mutex_wait_us":221,"thread_start_us":12820,"threads_started":1}
I20260810 06:54:54.864065 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.841136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45418 (local address 127.22.11.131:46763)
0810 06:54:54.852516 (+ 11380us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.852523 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.853808 (+ 1285us) server_negotiation.cc:407] Connection header received
0810 06:54:54.853899 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.853905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.854118 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.854316 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:54.855421 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.856935 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.859833 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.861163 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.861294 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.863488 (+ 2194us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.863584 (+ 96us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.863728 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11205,"thread_start_us":11131,"threads_started":1}
I20260810 06:54:54.865561 23369 tablet_service.cc:1467] Processing CreateTablet for tablet 66a7e7c52c25449a9592a9308d6e757d (DEFAULT_TABLE table=test-table [id=dc021af809ec47efa2060b6805e4b57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:54.865418 23167 tablet_service.cc:1467] Processing CreateTablet for tablet 66a7e7c52c25449a9592a9308d6e757d (DEFAULT_TABLE table=test-table [id=dc021af809ec47efa2060b6805e4b57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:54.866777 23369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a7e7c52c25449a9592a9308d6e757d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:54.866864 23167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a7e7c52c25449a9592a9308d6e757d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:54.867599 23579 tablet_service.cc:1467] Processing CreateTablet for tablet 66a7e7c52c25449a9592a9308d6e757d (DEFAULT_TABLE table=test-table [id=dc021af809ec47efa2060b6805e4b57f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:54.868969 23579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a7e7c52c25449a9592a9308d6e757d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:54.877527 23717 tablet_bootstrap.cc:492] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Bootstrap starting.
I20260810 06:54:54.880705 23717 tablet_bootstrap.cc:654] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:54.880839 23718 tablet_bootstrap.cc:492] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Bootstrap starting.
I20260810 06:54:54.882113 23717 log.cc:824] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:54.882356 23719 tablet_bootstrap.cc:492] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Bootstrap starting.
I20260810 06:54:54.884346 23718 tablet_bootstrap.cc:654] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:54.885422 23719 tablet_bootstrap.cc:654] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:54.885407 23718 log.cc:824] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:54.886478 23719 log.cc:824] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:54.887182 23717 tablet_bootstrap.cc:492] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: No bootstrap required, opened a new log
I20260810 06:54:54.887451 23717 ts_tablet_manager.cc:1397] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.001s
I20260810 06:54:54.888535 23718 tablet_bootstrap.cc:492] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: No bootstrap required, opened a new log
I20260810 06:54:54.888818 23718 ts_tablet_manager.cc:1397] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260810 06:54:54.888899 23719 tablet_bootstrap.cc:492] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: No bootstrap required, opened a new log
I20260810 06:54:54.889134 23719 ts_tablet_manager.cc:1397] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 06:54:54.896461 23719 raft_consensus.cc:348] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.896848 23719 raft_consensus.cc:374] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:54.896950 23719 raft_consensus.cc:729] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5be907b79ae4f7993809fc436df7350, State: Initialized, Role: FOLLOWER
I20260810 06:54:54.897538 23719 consensus_queue.cc:260] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.898245 23717 raft_consensus.cc:348] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.898559 23684 heartbeater.cc:503] Master 127.22.11.190:34609 was elected leader, sending a full tablet report...
I20260810 06:54:54.898641 23717 raft_consensus.cc:374] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:54.898736 23717 raft_consensus.cc:729] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe3a3dae31284cd28f5fdcb9e6f97794, State: Initialized, Role: FOLLOWER
I20260810 06:54:54.898976 23719 ts_tablet_manager.cc:1428] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260810 06:54:54.900287 23718 raft_consensus.cc:348] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.900730 23718 raft_consensus.cc:374] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:54.900840 23718 raft_consensus.cc:729] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5399f15cb3d45d1acbbeb936a9a0606, State: Initialized, Role: FOLLOWER
I20260810 06:54:54.901438 23718 consensus_queue.cc:260] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.902457 23273 heartbeater.cc:503] Master 127.22.11.190:34609 was elected leader, sending a full tablet report...
I20260810 06:54:54.903090 23718 ts_tablet_manager.cc:1428] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260810 06:54:54.903169 23275 tablet.cc:2369] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:54:54.905818 23717 consensus_queue.cc:260] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.910920 23476 heartbeater.cc:503] Master 127.22.11.190:34609 was elected leader, sending a full tablet report...
I20260810 06:54:54.911442 23717 ts_tablet_manager.cc:1428] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Time spent starting tablet: real 0.024s user 0.011s sys 0.007s
I20260810 06:54:54.966013 23726 raft_consensus.cc:482] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:54:54.966297 23726 raft_consensus.cc:504] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.967967 23726 leader_election.cc:288] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe3a3dae31284cd28f5fdcb9e6f97794 (127.22.11.130:34773), c5399f15cb3d45d1acbbeb936a9a0606 (127.22.11.129:36807)
I20260810 06:54:54.974080 23686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.968184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:34773 (local address 127.22.11.131:39855)
0810 06:54:54.968340 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:54.968359 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:54.968479 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:54.968928 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:54.968933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:54.968967 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:54.969186 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.969198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.970409 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.970413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.972299 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.972314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.973537 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.973543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.973671 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.973695 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:54:54.973764 (+ 69us) client_negotiation.cc:240] Negotiation successful
0810 06:54:54.973857 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260810 06:54:54.974159 23709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.968244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:39855 (local address 127.22.11.130:34773)
0810 06:54:54.968426 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.968432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.968452 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:54:54.968704 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.968709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.968782 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.968871 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:54.969327 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.970269 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.972449 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.973408 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.973557 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.973627 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.973802 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.973924 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:54:54.974836 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.968420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:59549 (local address 127.22.11.129:36807)
0810 06:54:54.968642 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:54.968647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:54.969066 (+ 419us) server_negotiation.cc:407] Connection header received
0810 06:54:54.969212 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:54.969216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:54.969292 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:54.969386 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:54.969868 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.970739 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.972949 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.974037 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.974193 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.974259 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:54.974536 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 06:54:54.974642 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:54:54.974882 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:54.968191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.22.11.131:59549)
0810 06:54:54.968959 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:54.968974 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:54.969090 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:54.969408 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:54.969412 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:54.969433 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:54.969722 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.969729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.970896 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.970902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.972778 (+ 1876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:54.972789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:54.974208 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:54.974213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:54.974360 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:54.974389 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 06:54:54.974476 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 06:54:54.974590 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":144,"thread_start_us":325,"threads_started":1}
I20260810 06:54:54.975308 23408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7e7c52c25449a9592a9308d6e757d" candidate_uuid: "c5be907b79ae4f7993809fc436df7350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" is_pre_election: true
I20260810 06:54:54.975854 23408 raft_consensus.cc:2454] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5be907b79ae4f7993809fc436df7350 in term 0.
I20260810 06:54:54.975855 23207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7e7c52c25449a9592a9308d6e757d" candidate_uuid: "c5be907b79ae4f7993809fc436df7350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" is_pre_election: true
I20260810 06:54:54.976389 23207 raft_consensus.cc:2454] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5be907b79ae4f7993809fc436df7350 in term 0.
I20260810 06:54:54.976503 23506 leader_election.cc:302] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5be907b79ae4f7993809fc436df7350, fe3a3dae31284cd28f5fdcb9e6f97794; no voters:
I20260810 06:54:54.976922 23726 raft_consensus.cc:2781] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:54:54.977072 23726 raft_consensus.cc:482] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:54:54.977173 23726 raft_consensus.cc:3037] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:54.979282 23726 raft_consensus.cc:504] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.979816 23726 leader_election.cc:288] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [CANDIDATE]: Term 1 election: Requested vote from peers fe3a3dae31284cd28f5fdcb9e6f97794 (127.22.11.130:34773), c5399f15cb3d45d1acbbeb936a9a0606 (127.22.11.129:36807)
I20260810 06:54:54.980208 23408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7e7c52c25449a9592a9308d6e757d" candidate_uuid: "c5be907b79ae4f7993809fc436df7350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794"
I20260810 06:54:54.980232 23207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a7e7c52c25449a9592a9308d6e757d" candidate_uuid: "c5be907b79ae4f7993809fc436df7350" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5399f15cb3d45d1acbbeb936a9a0606"
I20260810 06:54:54.980360 23408 raft_consensus.cc:3037] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:54.980405 23207 raft_consensus.cc:3037] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:54.982553 23207 raft_consensus.cc:2454] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5be907b79ae4f7993809fc436df7350 in term 1.
I20260810 06:54:54.982564 23408 raft_consensus.cc:2454] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5be907b79ae4f7993809fc436df7350 in term 1.
I20260810 06:54:54.982892 23506 leader_election.cc:302] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c5be907b79ae4f7993809fc436df7350, fe3a3dae31284cd28f5fdcb9e6f97794; no voters:
I20260810 06:54:54.983156 23726 raft_consensus.cc:2781] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:54.983562 23726 raft_consensus.cc:686] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 1 LEADER]: Becoming Leader. State: Replica: c5be907b79ae4f7993809fc436df7350, State: Running, Role: LEADER
I20260810 06:54:54.983994 23726 consensus_queue.cc:237] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } }
I20260810 06:54:54.988200 22963 catalog_manager.cc:5495] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5be907b79ae4f7993809fc436df7350 (127.22.11.131). New cstate: current_term: 1 leader_uuid: "c5be907b79ae4f7993809fc436df7350" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5be907b79ae4f7993809fc436df7350" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46763 } health_report { overall_health: HEALTHY } } }
I20260810 06:54:55.007676 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.002281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.22.11.131:46763)
0810 06:54:55.002460 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:55.002466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:55.002485 (+ 19us) server_negotiation.cc:407] Connection header received
0810 06:54:55.002705 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:55.002711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:55.002795 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:55.002917 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:54:55.003327 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.004050 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.005491 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.005910 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.006019 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:55.006599 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:54:55.007103 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:54:55.007260 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:55.007415 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 06:54:55.007456 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:54:55.015559 23207 raft_consensus.cc:1261] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 1 FOLLOWER]: Refusing update from remote peer c5be907b79ae4f7993809fc436df7350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:55.015560 23408 raft_consensus.cc:1261] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 1 FOLLOWER]: Refusing update from remote peer c5be907b79ae4f7993809fc436df7350: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:54:55.016426 23731 consensus_queue.cc:1035] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe3a3dae31284cd28f5fdcb9e6f97794" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 34773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:55.016795 23726 consensus_queue.cc:1035] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5399f15cb3d45d1acbbeb936a9a0606" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:55.026566 23735 mvcc.cc:205] Tried to move back new op lower bound from 7316868689974784000 to 7316868689858195456. Current Snapshot: MvccSnapshot[applied={T|T < 7316868689974784000}]
I20260810 06:54:55.031399 23737 mvcc.cc:205] Tried to move back new op lower bound from 7316868689974784000 to 7316868689858195456. Current Snapshot: MvccSnapshot[applied={T|T < 7316868689974784000}]
I20260810 06:54:55.040872 22963 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:55.041165 22963 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54738:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:55.044759 22963 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 776d0f61db0442b0b4d9904a8564ac4d: Sending DeleteTablet for 3 replicas of tablet 66a7e7c52c25449a9592a9308d6e757d
I20260810 06:54:55.045755 23167 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a7e7c52c25449a9592a9308d6e757d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:55 UTC) from {username='slave'} at 127.0.0.1:44998
I20260810 06:54:55.045794 23369 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a7e7c52c25449a9592a9308d6e757d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:55 UTC) from {username='slave'} at 127.0.0.1:32956
I20260810 06:54:55.045782 23579 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a7e7c52c25449a9592a9308d6e757d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:55 UTC) from {username='slave'} at 127.0.0.1:45418
I20260810 06:54:55.047832 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23067
I20260810 06:54:55.059428 23749 tablet_replica.cc:330] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: stopping tablet replica
I20260810 06:54:55.059841 23749 raft_consensus.cc:2229] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:55.060319 23749 raft_consensus.cc:2258] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:55.061043 23748 tablet_replica.cc:330] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: stopping tablet replica
I20260810 06:54:55.061417 23748 raft_consensus.cc:2229] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:55.061889 23748 raft_consensus.cc:2258] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:55.064016 23748 ts_tablet_manager.cc:1905] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:55.070159 23749 ts_tablet_manager.cc:1905] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:55.071579 23751 tablet_replica.cc:330] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: stopping tablet replica
I20260810 06:54:55.071962 23751 raft_consensus.cc:2229] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:54:55.072674 23751 raft_consensus.cc:2258] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:55.074731 23715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.049233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.22.11.129:36807)
0810 06:54:55.049406 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:55.049413 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:55.064820 (+ 15407us) server_negotiation.cc:407] Connection header received
0810 06:54:55.065263 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:55.065269 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:55.065359 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:55.065435 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:55.066053 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.067207 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.069301 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.069889 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.069982 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:55.072748 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:55.072762 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:55.072779 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:55.073509 (+ 730us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:55.073575 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:55.073602 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:55.073719 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:55.074216 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 06:54:55.074335 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:54:55.075477 23268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:55.076233 23751 ts_tablet_manager.cc:1905] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:55.076774 23749 ts_tablet_manager.cc:1918] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:55.076937 23749 log.cc:1196] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/66a7e7c52c25449a9592a9308d6e757d
I20260810 06:54:55.077502 23749 ts_tablet_manager.cc:1939] T 66a7e7c52c25449a9592a9308d6e757d P fe3a3dae31284cd28f5fdcb9e6f97794: Deleting consensus metadata
I20260810 06:54:55.079280 22948 catalog_manager.cc:4841] TS fe3a3dae31284cd28f5fdcb9e6f97794 (127.22.11.130:34773): tablet 66a7e7c52c25449a9592a9308d6e757d (table test-table [id=dc021af809ec47efa2060b6805e4b57f]) successfully deleted
I20260810 06:54:55.084092 23751 ts_tablet_manager.cc:1918] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:55.084335 23751 log.cc:1196] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/66a7e7c52c25449a9592a9308d6e757d
I20260810 06:54:55.084908 23751 ts_tablet_manager.cc:1939] T 66a7e7c52c25449a9592a9308d6e757d P c5be907b79ae4f7993809fc436df7350: Deleting consensus metadata
I20260810 06:54:55.086583 22946 catalog_manager.cc:4841] TS c5be907b79ae4f7993809fc436df7350 (127.22.11.131:46763): tablet 66a7e7c52c25449a9592a9308d6e757d (table test-table [id=dc021af809ec47efa2060b6805e4b57f]) successfully deleted
I20260810 06:54:55.167431 23748 ts_tablet_manager.cc:1918] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:55.167662 23748 log.cc:1196] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/66a7e7c52c25449a9592a9308d6e757d
I20260810 06:54:55.168138 23748 ts_tablet_manager.cc:1939] T 66a7e7c52c25449a9592a9308d6e757d P c5399f15cb3d45d1acbbeb936a9a0606: Deleting consensus metadata
I20260810 06:54:55.168267 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23067
W20260810 06:54:55.192255 22947 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.11.129:36807 (error 108)
I20260810 06:54:55.192382 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23278
I20260810 06:54:55.194192 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.193718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45028)
0810 06:54:55.193885 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.194009 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 06:54:55.194597 22947 catalog_manager.cc:4584] TS c5399f15cb3d45d1acbbeb936a9a0606 (127.22.11.129:36807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 66a7e7c52c25449a9592a9308d6e757d: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
I20260810 06:54:55.200780 23709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.193915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.22.11.130:34773)
0810 06:54:55.194109 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:55.194114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:55.194134 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:54:55.194201 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:55.194206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:55.194331 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:55.194464 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:55.195000 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.195754 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.197925 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.198318 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.198412 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:55.199494 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:55.199503 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:55.199518 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:55.200083 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:55.200129 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:55.200150 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:55.200236 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:55.200375 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 06:54:55.200461 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:54:55.201426 23468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:55.248983 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.248230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45038)
0810 06:54:55.248567 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.248723 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260810 06:54:55.250573 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.250165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45048)
0810 06:54:55.250330 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.250431 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 06:54:55.292303 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23278
I20260810 06:54:55.311327 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.310713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45058)
0810 06:54:55.310982 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.311113 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 06:54:55.312579 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.312168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45068)
0810 06:54:55.312334 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.312431 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 06:54:55.317337 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23480
I20260810 06:54:55.325115 23712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.318176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45438 (local address 127.22.11.131:46763)
0810 06:54:55.318375 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:55.318381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:55.318401 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:54:55.318460 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:55.318464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:55.318560 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:55.318646 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:55.319142 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.320445 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.321737 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:55.322423 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:55.322555 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:55.323451 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:55.323462 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:55.323486 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:54:55.324314 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:55.324380 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:54:55.324412 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:55.324522 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:55.324604 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:54:55.324744 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:54:55.325873 23673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:55.393069 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.392545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45076)
0810 06:54:55.392775 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.392882 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 06:54:55.394112 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.393690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45084)
0810 06:54:55.393893 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.393978 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 06:54:55.444918 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23480
I20260810 06:54:55.473074 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 22918
I20260810 06:54:55.474202 23019 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:55.580812 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:55.580162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36807 (local address 127.0.0.1:45098)
0810 06:54:55.580435 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:55.580562 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:36807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260810 06:54:55.581050 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 22918
2026-08-10T06:54:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2735 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:54:55 dist-test-slave-f7th krb5kdc[23766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:54:55 dist-test-slave-f7th krb5kdc[23766](info): set up 2 sockets
Aug 10 06:54:55 dist-test-slave-f7th krb5kdc[23766](info): commencing operation
krb5kdc: starting...
W20260810 06:54:56.846257 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:54:57 dist-test-slave-f7th krb5kdc[23766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:54:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:54:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:54:57.127132 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39133
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:39133
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:57.455606 23783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:57.455891 23783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:57.456037 23783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:57.464174 23783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:54:57.464284 23783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:57.464362 23783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:57.464438 23783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:54:57.464498 23783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:54:57.474084 23783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:39133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:57.475301 23783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:57.476536 23783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:57.487649 23791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:57.487617 23792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:57.487623 23794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:57.488457 23783 server_base.cc:1034] running on GCE node
I20260810 06:54:57.489210 23783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:57.490466 23783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:57.491671 23783 hybrid_clock.cc:648] HybridClock initialized: now 1786344897491617 us; error 36 us; skew 500 ppm
I20260810 06:54:57.492005 23783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:57 dist-test-slave-f7th krb5kdc[23766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344897, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:57.494908 23783 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:54:57.496249 23783 webserver.cc:460] Webserver started at http://127.22.11.190:45945/ using document root <none> and password file <none>
I20260810 06:54:57.496937 23783 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:57.497069 23783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:57.497349 23783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:57.500159 23783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "17472409aff34fc0abd04e8b4e41ccee"
format_stamp: "Formatted at 2026-08-10 06:54:57 on dist-test-slave-f7th"
server_key: "6aca24a435e24a6695bacd403b226464"
server_key_iv: "5823302144c377c8d6e9b9201ab6a5e6"
server_key_version: "encryptionkey@0"
I20260810 06:54:57.500911 23783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "17472409aff34fc0abd04e8b4e41ccee"
format_stamp: "Formatted at 2026-08-10 06:54:57 on dist-test-slave-f7th"
server_key: "6aca24a435e24a6695bacd403b226464"
server_key_iv: "5823302144c377c8d6e9b9201ab6a5e6"
server_key_version: "encryptionkey@0"
I20260810 06:54:57.505775 23783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:54:57.509066 23810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:57.510313 23783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:54:57.510495 23783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "17472409aff34fc0abd04e8b4e41ccee"
format_stamp: "Formatted at 2026-08-10 06:54:57 on dist-test-slave-f7th"
server_key: "6aca24a435e24a6695bacd403b226464"
server_key_iv: "5823302144c377c8d6e9b9201ab6a5e6"
server_key_version: "encryptionkey@0"
I20260810 06:54:57.510666 23783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:57.521075 23783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:57.524580 23783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:57.524885 23783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:57.542246 23783 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:39133
I20260810 06:54:57.542249 23892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:39133 every 8 connection(s)
I20260810 06:54:57.543601 23783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:54:57.547248 23894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:57.552356 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23783
I20260810 06:54:57.552544 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:54:57.553021 22574 external_mini_cluster.cc:1428] Setting key 40e00e8e1fc8604cbf90e76a11084e4e
I20260810 06:54:57.557533 23894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee: Bootstrap starting.
I20260810 06:54:57.561050 23894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:57.562245 23894 log.cc:824] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:57.565387 23894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee: No bootstrap required, opened a new log
Aug 10 06:54:57 dist-test-slave-f7th krb5kdc[23766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:57.574203 23894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17472409aff34fc0abd04e8b4e41ccee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39133 } }
I20260810 06:54:57.574643 23894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:57.574750 23894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17472409aff34fc0abd04e8b4e41ccee, State: Initialized, Role: FOLLOWER
I20260810 06:54:57.575331 23894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17472409aff34fc0abd04e8b4e41ccee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39133 } }
I20260810 06:54:57.575573 23894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:54:57.575716 23894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:54:57.575875 23894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:57.576056 23899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:57.554763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.22.11.190:39133)
0810 06:54:57.555642 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:57.555657 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:57.555703 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:54:57.557376 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:57.557400 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:57.557774 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:57.558089 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:57.559884 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:57.561253 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:57.563041 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:57.563898 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:57.564018 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:57.568872 (+ 4854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:57.568880 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:57.568893 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:57.568929 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:57.571977 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:57.572820 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:57.572827 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:57.572832 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:57.573004 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:57.573377 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:57.573380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:57.573382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:57.573869 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:57.574084 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:57.574560 (+ 476us) server_negotiation.cc:299] Negotiation successful
0810 06:54:57.574966 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":377,"threads_started":1}
I20260810 06:54:57.578199 23894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17472409aff34fc0abd04e8b4e41ccee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39133 } }
I20260810 06:54:57.578701 23894 leader_election.cc:302] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17472409aff34fc0abd04e8b4e41ccee; no voters:
I20260810 06:54:57.579242 23894 leader_election.cc:288] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:54:57.579331 23901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:57.579977 23901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [term 1 LEADER]: Becoming Leader. State: Replica: 17472409aff34fc0abd04e8b4e41ccee, State: Running, Role: LEADER
I20260810 06:54:57.580471 23901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17472409aff34fc0abd04e8b4e41ccee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39133 } }
I20260810 06:54:57.581135 23894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:54:57.582959 23902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17472409aff34fc0abd04e8b4e41ccee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17472409aff34fc0abd04e8b4e41ccee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39133 } } }
I20260810 06:54:57.583266 23902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:57.584733 23906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17472409aff34fc0abd04e8b4e41ccee. Latest consensus state: current_term: 1 leader_uuid: "17472409aff34fc0abd04e8b4e41ccee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17472409aff34fc0abd04e8b4e41ccee" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39133 } } }
I20260810 06:54:57.585114 23906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee [sys.catalog]: This master's current role is: LEADER
I20260810 06:54:57.585275 23913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:54:57.595736 23913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:54:57.603924 23913 catalog_manager.cc:1347] Generated new cluster ID: 77ff0119faef4a9f9237849204c2b707
I20260810 06:54:57.604023 23913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:54:57.614452 23913 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:54:57.615806 23913 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:54:57.622766 23913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee: Generated new TSK 0
I20260810 06:54:57.623553 23913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.22.11.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
Entry for principal kudu/127.22.11.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab.
I20260810 06:54:57.697595 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39133
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:58.027166 23934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:58.027453 23934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:58.027594 23934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:58.034991 23934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:58.035112 23934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:58.035270 23934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:54:58.044819 23934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.129.keytab
--principal=kudu/127.22.11.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:58.046077 23934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:58.047361 23934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:58.059886 23941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:58.059924 23942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:58.059957 23944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:58.060964 23934 server_base.cc:1034] running on GCE node
I20260810 06:54:58.061511 23934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:58.062399 23934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:58.063601 23934 hybrid_clock.cc:648] HybridClock initialized: now 1786344898063552 us; error 38 us; skew 500 ppm
I20260810 06:54:58.063949 23934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:58 dist-test-slave-f7th krb5kdc[23766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344898, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:58.067015 23934 init.cc:377] Logged in from keytab as kudu/127.22.11.129@KRBTEST.COM (short username kudu)
I20260810 06:54:58.068344 23934 webserver.cc:460] Webserver started at http://127.22.11.129:37719/ using document root <none> and password file <none>
I20260810 06:54:58.069023 23934 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:58.069126 23934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:58.069375 23934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:58.072075 23934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "d930ea5766564f3fa8da3dec828080d5"
format_stamp: "Formatted at 2026-08-10 06:54:58 on dist-test-slave-f7th"
server_key: "74eda7ad323fc461540994d238836fd4"
server_key_iv: "6f0f88fb4fd9e1180e520c5fe74d4561"
server_key_version: "encryptionkey@0"
I20260810 06:54:58.072824 23934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "d930ea5766564f3fa8da3dec828080d5"
format_stamp: "Formatted at 2026-08-10 06:54:58 on dist-test-slave-f7th"
server_key: "74eda7ad323fc461540994d238836fd4"
server_key_iv: "6f0f88fb4fd9e1180e520c5fe74d4561"
server_key_version: "encryptionkey@0"
I20260810 06:54:58.077824 23934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 06:54:58.081110 23956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:58.082482 23934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:54:58.082615 23934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "d930ea5766564f3fa8da3dec828080d5"
format_stamp: "Formatted at 2026-08-10 06:54:58 on dist-test-slave-f7th"
server_key: "74eda7ad323fc461540994d238836fd4"
server_key_iv: "6f0f88fb4fd9e1180e520c5fe74d4561"
server_key_version: "encryptionkey@0"
I20260810 06:54:58.082768 23934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:58.094784 23934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:58.098395 23934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:58.098623 23934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:58.099453 23934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:58.100700 23934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:58.100772 23934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:58.100859 23934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:58.100901 23934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:58.130376 23934 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:36763
I20260810 06:54:58.130424 24132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:36763 every 8 connection(s)
I20260810 06:54:58.131654 23934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:54:58.136006 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 23934
I20260810 06:54:58.136253 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:54:58.136886 22574 external_mini_cluster.cc:1428] Setting key 5ec78d871815ee4b7e23bef812a945fe
Aug 10 06:54:58 dist-test-slave-f7th krb5kdc[23766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344898, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.129@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:58.151908 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:58.134122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:37607 (local address 127.22.11.190:39133)
0810 06:54:58.134506 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:58.134511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:58.135415 (+ 904us) server_negotiation.cc:407] Connection header received
0810 06:54:58.137806 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:58.137811 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:58.137882 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:58.138018 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:58.139756 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.140468 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.142479 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.142860 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.142948 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:58.146687 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:58.146692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:58.146695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:58.146727 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:58.149158 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:58.149911 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:58.149915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:58.149917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:58.150035 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:58.150437 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:58.150439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:58.150441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:58.150679 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:58.150783 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:58.151507 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 06:54:58.151663 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":194,"threads_started":1}
I20260810 06:54:58.153098 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:58.134562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39133 (local address 127.22.11.129:37607)
0810 06:54:58.135258 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:58.135295 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:58.137521 (+ 2226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:58.138193 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:58.138200 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:58.138795 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:58.139579 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:58.139605 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.140635 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.140640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:58.142355 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:58.142376 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.143069 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.143075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:58.143318 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:58.143962 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:58.143987 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:58.146471 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:58.149283 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:58.149292 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:58.149305 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:58.149817 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:58.150124 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:58.150128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:58.150131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:58.150350 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:58.150789 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:58.150796 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:58.151161 (+ 365us) client_negotiation.cc:769] Sending connection context
0810 06:54:58.151429 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 06:54:58.151887 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":266,"threads_started":1}
I20260810 06:54:58.154557 24133 heartbeater.cc:347] Connected to a master server at 127.22.11.190:39133
I20260810 06:54:58.154838 24133 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:58.155514 24133 heartbeater.cc:511] Master 127.22.11.190:39133 requested a full tablet report, sending...
I20260810 06:54:58.157510 23836 ts_manager.cc:194] Registered new tserver with Master: d930ea5766564f3fa8da3dec828080d5 (127.22.11.129:36763)
I20260810 06:54:58.158926 23836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.129@KRBTEST.COM'} at 127.22.11.129:37607
WARNING: no policy specified for kudu/127.22.11.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
Entry for principal kudu/127.22.11.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab.
I20260810 06:54:58.201223 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39133
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:58.523741 24146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:58.524061 24146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:58.524210 24146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:58.533636 24146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:58.533757 24146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:58.533932 24146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:54:58.543339 24146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.130.keytab
--principal=kudu/127.22.11.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:58.544440 24146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:58.545708 24146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:58.557926 24152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:58.558013 24153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:58.557979 24157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:58.558636 24146 server_base.cc:1034] running on GCE node
I20260810 06:54:58.559233 24146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:58.560098 24146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:58.561290 24146 hybrid_clock.cc:648] HybridClock initialized: now 1786344898561254 us; error 22 us; skew 500 ppm
I20260810 06:54:58.561589 24146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:58 dist-test-slave-f7th krb5kdc[23766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344898, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:58.564459 24146 init.cc:377] Logged in from keytab as kudu/127.22.11.130@KRBTEST.COM (short username kudu)
I20260810 06:54:58.565842 24146 webserver.cc:460] Webserver started at http://127.22.11.130:45081/ using document root <none> and password file <none>
I20260810 06:54:58.566427 24146 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:58.566534 24146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:58.566781 24146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:58.569556 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "a5c5153bbd944d939715ae5b1d983a40"
format_stamp: "Formatted at 2026-08-10 06:54:58 on dist-test-slave-f7th"
server_key: "5cfad7d63a5456f1c524382c4d3d9d80"
server_key_iv: "2be8cc16a4df5a032b46a90a88a52ccc"
server_key_version: "encryptionkey@0"
I20260810 06:54:58.570231 24146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "a5c5153bbd944d939715ae5b1d983a40"
format_stamp: "Formatted at 2026-08-10 06:54:58 on dist-test-slave-f7th"
server_key: "5cfad7d63a5456f1c524382c4d3d9d80"
server_key_iv: "2be8cc16a4df5a032b46a90a88a52ccc"
server_key_version: "encryptionkey@0"
I20260810 06:54:58.574959 24146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:54:58.578161 24168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:58.579313 24146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:54:58.579447 24146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "a5c5153bbd944d939715ae5b1d983a40"
format_stamp: "Formatted at 2026-08-10 06:54:58 on dist-test-slave-f7th"
server_key: "5cfad7d63a5456f1c524382c4d3d9d80"
server_key_iv: "2be8cc16a4df5a032b46a90a88a52ccc"
server_key_version: "encryptionkey@0"
I20260810 06:54:58.579617 24146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:58.591095 24146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:58.594548 24146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:58.594791 24146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:58.595683 24146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:58.596949 24146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:58.597021 24146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:58.597132 24146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:58.597182 24146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:58.627117 24146 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:38365
I20260810 06:54:58.627159 24351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:38365 every 8 connection(s)
I20260810 06:54:58.628324 24146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:54:58.636405 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24146
I20260810 06:54:58.636610 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:54:58.637210 22574 external_mini_cluster.cc:1428] Setting key 76d0fdfc107e7cdbef0e12066717b7aa
Aug 10 06:54:58 dist-test-slave-f7th krb5kdc[23766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344898, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.130@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:58.647552 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:58.630689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:40319 (local address 127.22.11.190:39133)
0810 06:54:58.630845 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:58.630850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:58.631940 (+ 1090us) server_negotiation.cc:407] Connection header received
0810 06:54:58.633737 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:58.633741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:58.633810 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:58.633895 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:58.635239 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.635935 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.638054 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.638586 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.638682 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:58.642309 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:58.642314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:58.642318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:58.642350 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:58.644670 (+ 2320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:58.645354 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:58.645358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:58.645360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:58.645485 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:58.645890 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:58.645894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:58.645896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:58.646166 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:58.646315 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:58.646980 (+ 665us) server_negotiation.cc:299] Negotiation successful
0810 06:54:58.647224 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 06:54:58.648470 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:58.631111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39133 (local address 127.22.11.130:40319)
0810 06:54:58.631772 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:58.631818 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:58.633484 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:58.634072 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:58.634080 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:58.634563 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:58.635109 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:58.635126 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.636063 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.636068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:58.637915 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:58.637932 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:58.638807 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:58.638815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:58.639035 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:58.639672 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:58.639701 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:58.642097 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:58.644791 (+ 2694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:58.644799 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:58.644815 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:58.645221 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:58.645589 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:58.645594 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:58.645598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:58.645775 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:58.646290 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:58.646297 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:58.646622 (+ 325us) client_negotiation.cc:769] Sending connection context
0810 06:54:58.646864 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 06:54:58.647378 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":248,"threads_started":1}
I20260810 06:54:58.649987 24355 heartbeater.cc:347] Connected to a master server at 127.22.11.190:39133
I20260810 06:54:58.650277 24355 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:58.650949 24355 heartbeater.cc:511] Master 127.22.11.190:39133 requested a full tablet report, sending...
I20260810 06:54:58.652420 23836 ts_manager.cc:194] Registered new tserver with Master: a5c5153bbd944d939715ae5b1d983a40 (127.22.11.130:38365)
I20260810 06:54:58.653273 23836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.130@KRBTEST.COM'} at 127.22.11.130:40319
WARNING: no policy specified for kudu/127.22.11.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
Entry for principal kudu/127.22.11.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab.
I20260810 06:54:58.700663 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39133
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:54:59.025797 24361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:54:59.026086 24361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:54:59.026225 24361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:54:59.033450 24361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:54:59.033568 24361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:54:59.033715 24361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:54:59.043210 24361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.131.keytab
--principal=kudu/127.22.11.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:39133
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:54:59.044497 24361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:54:59.045814 24361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:54:59.058080 24371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:59.058161 24369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:54:59.058131 24368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:54:59.059183 24361 server_base.cc:1034] running on GCE node
I20260810 06:54:59.059741 24361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:54:59.060578 24361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:54:59.061846 24361 hybrid_clock.cc:648] HybridClock initialized: now 1786344899061781 us; error 46 us; skew 500 ppm
I20260810 06:54:59.062194 24361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:54:59 dist-test-slave-f7th krb5kdc[23766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344899, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:54:59.065387 24361 init.cc:377] Logged in from keytab as kudu/127.22.11.131@KRBTEST.COM (short username kudu)
I20260810 06:54:59.066707 24361 webserver.cc:460] Webserver started at http://127.22.11.131:37343/ using document root <none> and password file <none>
I20260810 06:54:59.067317 24361 fs_manager.cc:359] Metadata directory not provided
I20260810 06:54:59.067425 24361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:54:59.067682 24361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:54:59.070586 24361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "5c695157e6994db79d8b38a3af8132dc"
format_stamp: "Formatted at 2026-08-10 06:54:59 on dist-test-slave-f7th"
server_key: "31f8023212529d3b54093bbd2c233117"
server_key_iv: "3b2b9095491691382e2e702f3a88e127"
server_key_version: "encryptionkey@0"
I20260810 06:54:59.071300 24361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "5c695157e6994db79d8b38a3af8132dc"
format_stamp: "Formatted at 2026-08-10 06:54:59 on dist-test-slave-f7th"
server_key: "31f8023212529d3b54093bbd2c233117"
server_key_iv: "3b2b9095491691382e2e702f3a88e127"
server_key_version: "encryptionkey@0"
I20260810 06:54:59.076246 24361 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:54:59.079627 24383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:59.080920 24361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:54:59.081089 24361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "5c695157e6994db79d8b38a3af8132dc"
format_stamp: "Formatted at 2026-08-10 06:54:59 on dist-test-slave-f7th"
server_key: "31f8023212529d3b54093bbd2c233117"
server_key_iv: "3b2b9095491691382e2e702f3a88e127"
server_key_version: "encryptionkey@0"
I20260810 06:54:59.081274 24361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:54:59.095997 24361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:54:59.100081 24361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:54:59.100356 24361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:54:59.101315 24361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:54:59.102607 24361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:54:59.102680 24361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:59.102837 24361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:54:59.102900 24361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:54:59.131644 24361 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:46601
I20260810 06:54:59.131685 24557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:46601 every 8 connection(s)
I20260810 06:54:59.132979 24361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:54:59.135943 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24361
I20260810 06:54:59.136109 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:54:59.136655 22574 external_mini_cluster.cc:1428] Setting key 1bd228183878b7117e23119706091b3d
Aug 10 06:54:59 dist-test-slave-f7th krb5kdc[23766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344899, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.131@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:54:59.152211 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.135588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:35921 (local address 127.22.11.190:39133)
0810 06:54:59.135806 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.135813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.137073 (+ 1260us) server_negotiation.cc:407] Connection header received
0810 06:54:59.139240 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.139245 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.139330 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.139421 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:59.140870 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.141614 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.143025 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.143401 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.143485 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.146696 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:59.146701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:59.146704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:59.146736 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:59.149151 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.149862 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.149866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.149868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.150003 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.150416 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.150420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.150422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.150751 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:59.150891 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.151792 (+ 901us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.151957 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:54:59.153000 24565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.136044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39133 (local address 127.22.11.131:35921)
0810 06:54:59.136892 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:59.136951 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:59.138931 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:59.139654 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:59.139662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:59.140150 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:54:59.140732 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.140751 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.141730 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.141734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.142920 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.142932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.143564 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.143569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.143751 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.144356 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:54:59.144378 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:54:59.146505 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:54:59.149281 (+ 2776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:59.149289 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:59.149304 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:59.149736 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:59.150114 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:54:59.150117 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:54:59.150120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:54:59.150299 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:54:59.150878 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:54:59.150885 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:54:59.151271 (+ 386us) client_negotiation.cc:769] Sending connection context
0810 06:54:59.151516 (+ 245us) client_negotiation.cc:240] Negotiation successful
0810 06:54:59.151901 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":296,"threads_started":1}
I20260810 06:54:59.154209 24559 heartbeater.cc:347] Connected to a master server at 127.22.11.190:39133
I20260810 06:54:59.154500 24559 heartbeater.cc:464] Registering TS with master...
I20260810 06:54:59.155169 24559 heartbeater.cc:511] Master 127.22.11.190:39133 requested a full tablet report, sending...
I20260810 06:54:59.156437 23836 ts_manager.cc:194] Registered new tserver with Master: 5c695157e6994db79d8b38a3af8132dc (127.22.11.131:46601)
I20260810 06:54:59.157224 23836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.22.11.131@KRBTEST.COM'} at 127.22.11.131:35921
I20260810 06:54:59.161731 24133 heartbeater.cc:503] Master 127.22.11.190:39133 was elected leader, sending a full tablet report...
I20260810 06:54:59.163761 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:54:59.179023 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.167699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.22.11.190:39133)
0810 06:54:59.167889 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.167895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.168335 (+ 440us) server_negotiation.cc:407] Connection header received
0810 06:54:59.168462 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.168466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.168536 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.168639 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:59.169970 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.170662 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.171852 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.172226 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.172309 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.174528 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:59.174533 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:59.174536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:59.174564 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:59.176892 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.177483 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.177486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.177497 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.177611 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.177885 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.177888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.177890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.178183 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:59.178354 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.178585 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.178740 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:54:59.182428 23836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:54:59.189613 23836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:54:59.211934 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.200857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:46601 (local address 127.0.0.1:56958)
0810 06:54:59.201562 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:59.201602 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:59.201780 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:59.203566 (+ 1786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:59.203575 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:59.203616 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:59.204026 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.204048 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.206411 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.206417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.208916 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.208933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.210891 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.210898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.211122 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.211184 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 06:54:59.211301 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:54:59.211492 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":311,"threads_started":1}
I20260810 06:54:59.213322 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.200983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56958 (local address 127.22.11.131:46601)
0810 06:54:59.202906 (+ 1923us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.202914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.202950 (+ 36us) server_negotiation.cc:407] Connection header received
0810 06:54:59.203073 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.203081 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.203327 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.204250 (+ 923us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:59.204524 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.206245 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.209160 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.210646 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.212429 (+ 1783us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.212707 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.212789 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.212959 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1746,"thread_start_us":1592,"threads_started":1}
I20260810 06:54:59.217487 24457 tablet_service.cc:1467] Processing CreateTablet for tablet 4f14cd1cb10b4658ae22611ab6246d7f (DEFAULT_TABLE table=test-table [id=25ad8819102843508d8276725de10a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:59.218909 24457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f14cd1cb10b4658ae22611ab6246d7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:59.220963 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.201991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36763 (local address 127.0.0.1:42744)
0810 06:54:59.212165 (+ 10174us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:59.212180 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:59.212268 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:59.214630 (+ 2362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:59.214636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:59.214685 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:59.214912 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.214923 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.216788 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.216793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.218406 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.218421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.220166 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.220172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.220341 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.220372 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 06:54:59.220457 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:54:59.220592 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10099,"mutex_wait_us":3,"spinlock_wait_cycles":1792,"thread_start_us":10141,"threads_started":1}
I20260810 06:54:59.221005 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.202624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.22.11.129:36763)
0810 06:54:59.213998 (+ 11374us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.214006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.214045 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:54:59.214143 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.214151 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.214398 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.215141 (+ 743us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:59.215384 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.216662 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.218599 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.220009 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.220131 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.220351 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.220509 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.220707 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11186,"thread_start_us":11034,"threads_started":1}
I20260810 06:54:59.222152 24594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.201298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:38365 (local address 127.0.0.1:41448)
0810 06:54:59.211837 (+ 10539us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:59.211858 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:59.212631 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:59.214636 (+ 2005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:59.214641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:59.214696 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:59.215484 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.215498 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.217302 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.217307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.219237 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.219252 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.221359 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.221365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.221526 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.221559 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:54:59.221644 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:54:59.221763 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10397,"mutex_wait_us":808,"spinlock_wait_cycles":11904,"thread_start_us":13120,"threads_started":1}
I20260810 06:54:59.222656 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.201642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.22.11.130:38365)
0810 06:54:59.202384 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.202394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.211983 (+ 9589us) server_negotiation.cc:407] Connection header received
0810 06:54:59.212851 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.212863 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.213138 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.213297 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:59.215684 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.217166 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.219457 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.220846 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.220979 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.221456 (+ 477us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.221714 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.222362 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":500,"threads_started":1}
I20260810 06:54:59.224493 24028 tablet_service.cc:1467] Processing CreateTablet for tablet 4f14cd1cb10b4658ae22611ab6246d7f (DEFAULT_TABLE table=test-table [id=25ad8819102843508d8276725de10a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:59.225700 24028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f14cd1cb10b4658ae22611ab6246d7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:59.226424 24243 tablet_service.cc:1467] Processing CreateTablet for tablet 4f14cd1cb10b4658ae22611ab6246d7f (DEFAULT_TABLE table=test-table [id=25ad8819102843508d8276725de10a5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:54:59.227802 24243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f14cd1cb10b4658ae22611ab6246d7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:54:59.233078 24597 tablet_bootstrap.cc:492] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Bootstrap starting.
I20260810 06:54:59.236688 24597 tablet_bootstrap.cc:654] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:59.238065 24597 log.cc:824] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:59.238790 24602 tablet_bootstrap.cc:492] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Bootstrap starting.
I20260810 06:54:59.241385 24605 tablet_bootstrap.cc:492] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Bootstrap starting.
I20260810 06:54:59.242486 24602 tablet_bootstrap.cc:654] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:59.243429 24602 log.cc:824] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:59.244787 24605 tablet_bootstrap.cc:654] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Neither blocks nor log segments found. Creating new log.
I20260810 06:54:59.245973 24605 log.cc:824] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Log is configured to *not* fsync() on all Append() calls
I20260810 06:54:59.246308 24597 tablet_bootstrap.cc:492] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: No bootstrap required, opened a new log
I20260810 06:54:59.246582 24597 ts_tablet_manager.cc:1397] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.003s
I20260810 06:54:59.246941 24602 tablet_bootstrap.cc:492] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: No bootstrap required, opened a new log
I20260810 06:54:59.247192 24602 ts_tablet_manager.cc:1397] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 06:54:59.255271 24605 tablet_bootstrap.cc:492] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: No bootstrap required, opened a new log
I20260810 06:54:59.255542 24605 ts_tablet_manager.cc:1397] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260810 06:54:59.256839 24597 raft_consensus.cc:348] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.257292 24597 raft_consensus.cc:374] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:59.257390 24597 raft_consensus.cc:729] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c695157e6994db79d8b38a3af8132dc, State: Initialized, Role: FOLLOWER
I20260810 06:54:59.258008 24597 consensus_queue.cc:260] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.258186 24602 raft_consensus.cc:348] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.258585 24602 raft_consensus.cc:374] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:59.258694 24602 raft_consensus.cc:729] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d930ea5766564f3fa8da3dec828080d5, State: Initialized, Role: FOLLOWER
I20260810 06:54:59.259224 24559 heartbeater.cc:503] Master 127.22.11.190:39133 was elected leader, sending a full tablet report...
I20260810 06:54:59.259279 24602 consensus_queue.cc:260] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.259835 24597 ts_tablet_manager.cc:1428] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260810 06:54:59.261575 24602 ts_tablet_manager.cc:1428] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260810 06:54:59.265501 24605 raft_consensus.cc:348] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.265885 24605 raft_consensus.cc:374] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:54:59.265982 24605 raft_consensus.cc:729] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5c5153bbd944d939715ae5b1d983a40, State: Initialized, Role: FOLLOWER
I20260810 06:54:59.266479 24605 consensus_queue.cc:260] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.267390 24355 heartbeater.cc:503] Master 127.22.11.190:39133 was elected leader, sending a full tablet report...
I20260810 06:54:59.267961 24605 ts_tablet_manager.cc:1428] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20260810 06:54:59.336090 24613 raft_consensus.cc:482] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:54:59.336402 24613 raft_consensus.cc:504] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.338179 24613 leader_election.cc:288] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c695157e6994db79d8b38a3af8132dc (127.22.11.131:46601), d930ea5766564f3fa8da3dec828080d5 (127.22.11.129:36763)
I20260810 06:54:59.344646 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.338651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:36773 (local address 127.22.11.131:46601)
0810 06:54:59.338837 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.338842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.339319 (+ 477us) server_negotiation.cc:407] Connection header received
0810 06:54:59.339550 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.339555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.339633 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.339728 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:59.340386 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.341203 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.342979 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.343732 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.343879 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.343953 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.344253 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.344377 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:54:59.344806 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.338510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:46601 (local address 127.22.11.130:36773)
0810 06:54:59.339214 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:59.339232 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:59.339384 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:59.339863 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:59.339869 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:59.339897 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:59.340243 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.340251 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.341332 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.341337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.342822 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.342835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.343948 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.343955 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.344098 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.344122 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:54:59.344214 (+ 92us) client_negotiation.cc:240] Negotiation successful
0810 06:54:59.344366 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"mutex_wait_us":1,"spinlock_wait_cycles":15616,"thread_start_us":374,"threads_started":1}
I20260810 06:54:59.344930 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.339554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:49129 (local address 127.22.11.129:36763)
0810 06:54:59.339763 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.339769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.340026 (+ 257us) server_negotiation.cc:407] Connection header received
0810 06:54:59.340197 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.340202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.340258 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.340329 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:54:59.340877 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.341607 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.343266 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.344010 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.344155 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.344226 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.344566 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.344712 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:54:59.345106 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.339340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:36763 (local address 127.22.11.130:49129)
0810 06:54:59.339919 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 06:54:59.339932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:54:59.340066 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:54:59.340358 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:54:59.340361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:54:59.340384 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:54:59.340717 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.340727 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.341750 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.341755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.343116 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:54:59.343129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.344215 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.344221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:54:59.344414 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.344438 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 06:54:59.344523 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 06:54:59.344678 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":33,"thread_start_us":301,"threads_started":1}
I20260810 06:54:59.346011 24489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f14cd1cb10b4658ae22611ab6246d7f" candidate_uuid: "a5c5153bbd944d939715ae5b1d983a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c695157e6994db79d8b38a3af8132dc" is_pre_election: true
I20260810 06:54:59.346019 24056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f14cd1cb10b4658ae22611ab6246d7f" candidate_uuid: "a5c5153bbd944d939715ae5b1d983a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d930ea5766564f3fa8da3dec828080d5" is_pre_election: true
I20260810 06:54:59.346465 24489 raft_consensus.cc:2454] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5c5153bbd944d939715ae5b1d983a40 in term 0.
I20260810 06:54:59.346468 24056 raft_consensus.cc:2454] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5c5153bbd944d939715ae5b1d983a40 in term 0.
I20260810 06:54:59.347070 24169 leader_election.cc:302] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5c5153bbd944d939715ae5b1d983a40, d930ea5766564f3fa8da3dec828080d5; no voters:
I20260810 06:54:59.347414 24613 raft_consensus.cc:2781] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:54:59.347539 24613 raft_consensus.cc:482] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:54:59.347637 24613 raft_consensus.cc:3037] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:59.349831 24613 raft_consensus.cc:504] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.350376 24613 leader_election.cc:288] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [CANDIDATE]: Term 1 election: Requested vote from peers 5c695157e6994db79d8b38a3af8132dc (127.22.11.131:46601), d930ea5766564f3fa8da3dec828080d5 (127.22.11.129:36763)
I20260810 06:54:59.350780 24489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f14cd1cb10b4658ae22611ab6246d7f" candidate_uuid: "a5c5153bbd944d939715ae5b1d983a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c695157e6994db79d8b38a3af8132dc"
I20260810 06:54:59.350831 24056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f14cd1cb10b4658ae22611ab6246d7f" candidate_uuid: "a5c5153bbd944d939715ae5b1d983a40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d930ea5766564f3fa8da3dec828080d5"
I20260810 06:54:59.350946 24489 raft_consensus.cc:3037] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:59.350996 24056 raft_consensus.cc:3037] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:54:59.353097 24489 raft_consensus.cc:2454] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5c5153bbd944d939715ae5b1d983a40 in term 1.
I20260810 06:54:59.353142 24056 raft_consensus.cc:2454] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5c5153bbd944d939715ae5b1d983a40 in term 1.
I20260810 06:54:59.353451 24169 leader_election.cc:302] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c695157e6994db79d8b38a3af8132dc, a5c5153bbd944d939715ae5b1d983a40; no voters:
I20260810 06:54:59.353767 24613 raft_consensus.cc:2781] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:54:59.354172 24613 raft_consensus.cc:686] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 1 LEADER]: Becoming Leader. State: Replica: a5c5153bbd944d939715ae5b1d983a40, State: Running, Role: LEADER
I20260810 06:54:59.354605 24613 consensus_queue.cc:237] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } }
I20260810 06:54:59.359136 23834 catalog_manager.cc:5495] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5c5153bbd944d939715ae5b1d983a40 (127.22.11.130). New cstate: current_term: 1 leader_uuid: "a5c5153bbd944d939715ae5b1d983a40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5c5153bbd944d939715ae5b1d983a40" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 38365 } health_report { overall_health: HEALTHY } } }
I20260810 06:54:59.376192 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.371172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.22.11.130:38365)
0810 06:54:59.371349 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.371355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.371372 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:54:59.371562 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.371566 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.371637 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.371758 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:54:59.372124 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.372865 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.374208 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.374609 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.374714 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.375163 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:54:59.375632 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:54:59.375761 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.375850 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.375928 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 06:54:59.380203 24356 tablet.cc:2369] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:54:59.383910 24489 raft_consensus.cc:1261] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 1 FOLLOWER]: Refusing update from remote peer a5c5153bbd944d939715ae5b1d983a40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260810 06:54:59.383955 24134 tablet.cc:2369] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:54:59.383958 24056 raft_consensus.cc:1261] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 1 FOLLOWER]: Refusing update from remote peer a5c5153bbd944d939715ae5b1d983a40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260810 06:54:59.384478 24562 tablet.cc:2369] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:54:59.384958 24618 consensus_queue.cc:1035] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c695157e6994db79d8b38a3af8132dc" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 46601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:59.385396 24613 consensus_queue.cc:1035] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d930ea5766564f3fa8da3dec828080d5" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 36763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:54:59.395684 24623 mvcc.cc:205] Tried to move back new op lower bound from 7316868707866767360 to 7316868707760095232. Current Snapshot: MvccSnapshot[applied={T|T < 7316868707866767360}]
I20260810 06:54:59.396777 24622 mvcc.cc:205] Tried to move back new op lower bound from 7316868707866767360 to 7316868707760095232. Current Snapshot: MvccSnapshot[applied={T|T < 7316868707866767360 or (T in {7316868707866767360})}]
I20260810 06:54:59.417352 23831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:59.417619 23831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:54:59.421059 23831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17472409aff34fc0abd04e8b4e41ccee: Sending DeleteTablet for 3 replicas of tablet 4f14cd1cb10b4658ae22611ab6246d7f
I20260810 06:54:59.422087 24457 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f14cd1cb10b4658ae22611ab6246d7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:59 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:56958
I20260810 06:54:59.422241 24028 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f14cd1cb10b4658ae22611ab6246d7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:59 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:42744
I20260810 06:54:59.422371 24243 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f14cd1cb10b4658ae22611ab6246d7f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:54:59 UTC) from {username='kudu', principal='kudu/127.22.11.190@KRBTEST.COM'} at 127.0.0.1:41448
I20260810 06:54:59.423525 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23934
I20260810 06:54:59.434403 24635 tablet_replica.cc:330] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: stopping tablet replica
I20260810 06:54:59.434780 24635 raft_consensus.cc:2229] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:59.435232 24635 raft_consensus.cc:2258] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:59.438936 24635 ts_tablet_manager.cc:1905] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:59.438964 24634 tablet_replica.cc:330] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: stopping tablet replica
I20260810 06:54:59.439335 24634 raft_consensus.cc:2229] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:54:59.439781 24634 raft_consensus.cc:2258] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:59.440549 24636 tablet_replica.cc:330] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: stopping tablet replica
I20260810 06:54:59.440932 24636 raft_consensus.cc:2229] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:54:59.441553 24636 raft_consensus.cc:2258] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:54:59.443625 24634 ts_tablet_manager.cc:1905] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:59.446907 24635 ts_tablet_manager.cc:1918] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:59.447134 24635 log.cc:1196] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-0/wal/wals/4f14cd1cb10b4658ae22611ab6246d7f
I20260810 06:54:59.447805 24635 ts_tablet_manager.cc:1939] T 4f14cd1cb10b4658ae22611ab6246d7f P d930ea5766564f3fa8da3dec828080d5: Deleting consensus metadata
Aug 10 06:54:59 dist-test-slave-f7th krb5kdc[23766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.129@KRBTEST.COM
I20260810 06:54:59.449803 23811 catalog_manager.cc:4841] TS d930ea5766564f3fa8da3dec828080d5 (127.22.11.129:36763): tablet 4f14cd1cb10b4658ae22611ab6246d7f (table test-table [id=25ad8819102843508d8276725de10a5e]) successfully deleted
I20260810 06:54:59.450309 24634 ts_tablet_manager.cc:1918] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:59.450445 24634 log.cc:1196] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/4f14cd1cb10b4658ae22611ab6246d7f
I20260810 06:54:59.450429 24636 ts_tablet_manager.cc:1905] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:54:59.450865 24634 ts_tablet_manager.cc:1939] T 4f14cd1cb10b4658ae22611ab6246d7f P 5c695157e6994db79d8b38a3af8132dc: Deleting consensus metadata
I20260810 06:54:59.452128 23811 catalog_manager.cc:4841] TS 5c695157e6994db79d8b38a3af8132dc (127.22.11.131:46601): tablet 4f14cd1cb10b4658ae22611ab6246d7f (table test-table [id=25ad8819102843508d8276725de10a5e]) successfully deleted
I20260810 06:54:59.457661 24636 ts_tablet_manager.cc:1918] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:54:59.457844 24636 log.cc:1196] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/4f14cd1cb10b4658ae22611ab6246d7f
I20260810 06:54:59.458395 24636 ts_tablet_manager.cc:1939] T 4f14cd1cb10b4658ae22611ab6246d7f P a5c5153bbd944d939715ae5b1d983a40: Deleting consensus metadata
I20260810 06:54:59.459174 24595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.424954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.22.11.129:36763)
0810 06:54:59.425136 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.425142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.432033 (+ 6891us) server_negotiation.cc:407] Connection header received
0810 06:54:59.432227 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.432232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.432316 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.432419 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:59.440394 (+ 7975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.441651 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.443554 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.444265 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.444371 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.453240 (+ 8869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:59.453251 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:59.453265 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:59.453305 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:59.456119 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.456834 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.456842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.456844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.456990 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.457438 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.457443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.457447 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.457851 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:59.458114 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.458658 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.458869 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:54:59.459822 24127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:59.460317 23812 catalog_manager.cc:4841] TS a5c5153bbd944d939715ae5b1d983a40 (127.22.11.130:38365): tablet 4f14cd1cb10b4658ae22611ab6246d7f (table test-table [id=25ad8819102843508d8276725de10a5e]) successfully deleted
I20260810 06:54:59.552419 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23934
I20260810 06:54:59.584050 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24146
Aug 10 06:54:59 dist-test-slave-f7th krb5kdc[23766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.130@KRBTEST.COM
I20260810 06:54:59.601374 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.584944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.22.11.130:38365)
0810 06:54:59.585150 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.585156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.585179 (+ 23us) server_negotiation.cc:407] Connection header received
0810 06:54:59.585274 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.585280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.585382 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.585498 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:59.586911 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.588069 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.589487 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.590147 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.590283 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.594815 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:59.594832 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:59.594852 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:59.594899 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:59.598770 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.599363 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.599373 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.599376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.599539 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.599839 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.599844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.599847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.600259 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:59.600585 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.600767 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.600987 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:54:59.602212 24342 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:59.712769 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24146
I20260810 06:54:59.738024 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24361
Aug 10 06:54:59 dist-test-slave-f7th krb5kdc[23766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.131@KRBTEST.COM
I20260810 06:54:59.751869 24582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:54:59.738754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56966 (local address 127.22.11.131:46601)
0810 06:54:59.738912 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 06:54:59.738917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:54:59.738949 (+ 32us) server_negotiation.cc:407] Connection header received
0810 06:54:59.739123 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:54:59.739128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:54:59.739191 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:54:59.739280 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:54:59.740479 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.741246 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.742483 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:54:59.742856 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:54:59.742943 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:54:59.747100 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:54:59.747116 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:54:59.747132 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:54:59.747164 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:54:59.749735 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.750306 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.750312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.750315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.750445 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:54:59.750719 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:54:59.750722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:54:59.750724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:54:59.751052 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:54:59.751254 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 06:54:59.751472 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 06:54:59.751634 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 06:54:59.752518 24552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:59.840031 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24361
I20260810 06:54:59.867141 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 23783
I20260810 06:54:59.868180 23887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:54:59.977312 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 23783
2026-08-10T06:55:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4396 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T06:55:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:00Z Disabled control of system clock
I20260810 06:55:00.074045 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:35111
--webserver_interface=127.22.11.190
--webserver_port=0
--builtin_ntp_servers=127.22.11.148:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:35111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:55:00.399801 24651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:00.400076 24651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:00.400203 24651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:00.407248 24651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:00.407351 24651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:00.407414 24651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:00.407495 24651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:00.407557 24651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:00.416333 24651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:35111
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:35111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:00.417595 24651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:00.419360 24651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:00.430357 24660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:00.430516 24659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:00.430691 24663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:00.431099 24651 server_base.cc:1034] running on GCE node
I20260810 06:55:00.431789 24651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:00.433120 24651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:00.434311 24651 hybrid_clock.cc:648] HybridClock initialized: now 1786344900434263 us; error 39 us; skew 500 ppm
I20260810 06:55:00.434631 24651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:55:00.436801 24651 webserver.cc:460] Webserver started at http://127.22.11.190:45179/ using document root <none> and password file <none>
I20260810 06:55:00.437566 24651 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:00.437680 24651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:00.438021 24651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:00.441181 24651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "cd7914f0fd9546a3b263210f6d9f4775"
format_stamp: "Formatted at 2026-08-10 06:55:00 on dist-test-slave-f7th"
server_key: "b442dfe4116026625fb4934d1e261e84"
server_key_iv: "05ddc0a6fa1f81112b0fccb9388e1e4b"
server_key_version: "encryptionkey@0"
I20260810 06:55:00.442003 24651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "cd7914f0fd9546a3b263210f6d9f4775"
format_stamp: "Formatted at 2026-08-10 06:55:00 on dist-test-slave-f7th"
server_key: "b442dfe4116026625fb4934d1e261e84"
server_key_iv: "05ddc0a6fa1f81112b0fccb9388e1e4b"
server_key_version: "encryptionkey@0"
I20260810 06:55:00.446780 24651 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:55:00.449934 24672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:00.451120 24651 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:55:00.451282 24651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "cd7914f0fd9546a3b263210f6d9f4775"
format_stamp: "Formatted at 2026-08-10 06:55:00 on dist-test-slave-f7th"
server_key: "b442dfe4116026625fb4934d1e261e84"
server_key_iv: "05ddc0a6fa1f81112b0fccb9388e1e4b"
server_key_version: "encryptionkey@0"
I20260810 06:55:00.451464 24651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:00.461701 24651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:00.462428 24651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:00.462699 24651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:00.480320 24651 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:35111
I20260810 06:55:00.480326 24770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:35111 every 8 connection(s)
I20260810 06:55:00.481599 24651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:00.485160 24771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:00.487159 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24651
I20260810 06:55:00.487336 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:00.487839 22574 external_mini_cluster.cc:1428] Setting key 9e68f5ce3b4a0c48759eb967340c34ae
I20260810 06:55:00.496663 24771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775: Bootstrap starting.
I20260810 06:55:00.499944 24771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:00.500787 24775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:00.489683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35742 (local address 127.22.11.190:35111)
0810 06:55:00.490629 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:00.490639 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:00.490682 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:55:00.492301 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:00.492322 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:00.492772 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:00.493080 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:00.493634 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:00.494890 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:00.496656 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:00.497333 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:00.497462 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:00.498633 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:00.498640 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:00.498651 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:00.499127 (+ 476us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:00.499214 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:00.499234 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:00.499313 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:00.499534 (+ 221us) server_negotiation.cc:299] Negotiation successful
0810 06:55:00.499819 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":371,"threads_started":1}
I20260810 06:55:00.500993 24771 log.cc:824] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:00.503944 24771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775: No bootstrap required, opened a new log
I20260810 06:55:00.510761 24771 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7914f0fd9546a3b263210f6d9f4775" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35111 } }
I20260810 06:55:00.511080 24771 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:00.511176 24771 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd7914f0fd9546a3b263210f6d9f4775, State: Initialized, Role: FOLLOWER
I20260810 06:55:00.511696 24771 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7914f0fd9546a3b263210f6d9f4775" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35111 } }
I20260810 06:55:00.511914 24771 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:00.512034 24771 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:00.512174 24771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:00.514205 24771 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7914f0fd9546a3b263210f6d9f4775" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35111 } }
I20260810 06:55:00.514669 24771 leader_election.cc:302] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd7914f0fd9546a3b263210f6d9f4775; no voters:
I20260810 06:55:00.515190 24771 leader_election.cc:288] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:00.515298 24777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:00.516072 24777 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [term 1 LEADER]: Becoming Leader. State: Replica: cd7914f0fd9546a3b263210f6d9f4775, State: Running, Role: LEADER
I20260810 06:55:00.516510 24777 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7914f0fd9546a3b263210f6d9f4775" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35111 } }
I20260810 06:55:00.517069 24771 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:00.518975 24779 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd7914f0fd9546a3b263210f6d9f4775. Latest consensus state: current_term: 1 leader_uuid: "cd7914f0fd9546a3b263210f6d9f4775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7914f0fd9546a3b263210f6d9f4775" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35111 } } }
I20260810 06:55:00.519294 24779 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:00.520710 24778 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd7914f0fd9546a3b263210f6d9f4775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd7914f0fd9546a3b263210f6d9f4775" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35111 } } }
I20260810 06:55:00.521027 24778 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:00.521220 24783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:00.526407 24783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:00.534593 24783 catalog_manager.cc:1347] Generated new cluster ID: 6f2c403254a6405e8ad657409ee77cf4
I20260810 06:55:00.534699 24783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:00.547803 24783 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:00.549019 24783 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:00.558195 24783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775: Generated new TSK 0
I20260810 06:55:00.558911 24783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:00.573187 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.129:0
--local_ip_for_outbound_sockets=127.22.11.129
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:35111
--builtin_ntp_servers=127.22.11.148:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 06:55:00.920691 24806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:00.920948 24806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:00.921082 24806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:00.928239 24806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:00.928335 24806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:00.928488 24806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.129
I20260810 06:55:00.937330 24806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.22.11.129
--webserver_port=0
--tserver_master_addrs=127.22.11.190:35111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.129
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:00.938515 24806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:00.940243 24806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:00.952736 24814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:00.952795 24813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:00.952708 24822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:00.953858 24806 server_base.cc:1034] running on GCE node
I20260810 06:55:00.954471 24806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:00.955365 24806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:00.956562 24806 hybrid_clock.cc:648] HybridClock initialized: now 1786344900956490 us; error 63 us; skew 500 ppm
I20260810 06:55:00.956881 24806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:55:00.958959 24806 webserver.cc:460] Webserver started at http://127.22.11.129:33891/ using document root <none> and password file <none>
I20260810 06:55:00.959607 24806 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:00.959709 24806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:00.960049 24806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:00.962809 24806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data/instance:
uuid: "6b1923979c034041b4cc337671053a45"
format_stamp: "Formatted at 2026-08-10 06:55:00 on dist-test-slave-f7th"
server_key: "9e64a51a1db9390b8b6abc7700e86379"
server_key_iv: "0d6b4651a8b417c47e1627d88b007da5"
server_key_version: "encryptionkey@0"
I20260810 06:55:00.963563 24806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance:
uuid: "6b1923979c034041b4cc337671053a45"
format_stamp: "Formatted at 2026-08-10 06:55:00 on dist-test-slave-f7th"
server_key: "9e64a51a1db9390b8b6abc7700e86379"
server_key_iv: "0d6b4651a8b417c47e1627d88b007da5"
server_key_version: "encryptionkey@0"
I20260810 06:55:00.968327 24806 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260810 06:55:00.971544 24832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:00.972808 24806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:55:00.972954 24806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/wal
uuid: "6b1923979c034041b4cc337671053a45"
format_stamp: "Formatted at 2026-08-10 06:55:00 on dist-test-slave-f7th"
server_key: "9e64a51a1db9390b8b6abc7700e86379"
server_key_iv: "0d6b4651a8b417c47e1627d88b007da5"
server_key_version: "encryptionkey@0"
I20260810 06:55:00.973138 24806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:00.996961 24806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:00.997745 24806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:00.998031 24806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:00.998952 24806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:55:01.000401 24806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:55:01.000505 24806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:01.000610 24806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:55:01.000702 24806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 06:55:01.032078 24806 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.129:37157
I20260810 06:55:01.032132 25024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.129:37157 every 8 connection(s)
I20260810 06:55:01.033342 24806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/data/info.pb
I20260810 06:55:01.034384 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 24806
I20260810 06:55:01.034578 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-0/wal/instance
I20260810 06:55:01.035135 22574 external_mini_cluster.cc:1428] Setting key b44e8f3037931321a140965d2ac24953
I20260810 06:55:01.038187 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.130:0
--local_ip_for_outbound_sockets=127.22.11.130
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:35111
--builtin_ntp_servers=127.22.11.148:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:55:01.046242 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.036076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.129:60761 (local address 127.22.11.190:35111)
0810 06:55:01.036520 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:01.036525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:01.037893 (+ 1368us) server_negotiation.cc:407] Connection header received
0810 06:55:01.039996 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:01.040002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:01.040096 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:01.040206 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:01.041248 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.042202 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.043643 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.044153 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.044272 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:01.045480 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:01.045485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:01.045490 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:01.045510 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:01.045564 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:01.045579 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:01.045663 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:01.045828 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 06:55:01.045929 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":245,"threads_started":1}
I20260810 06:55:01.047055 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.036647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:35111 (local address 127.22.11.129:60761)
0810 06:55:01.037728 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:01.037780 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:01.039747 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:01.040381 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:01.040389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:01.040476 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:01.041091 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:01.041109 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.042324 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.042328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:01.043503 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:01.043514 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.044329 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.044333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:01.044514 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:01.045172 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:55:01.045174 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:01.045211 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:55:01.045290 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:01.045657 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:01.045663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:01.045668 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 06:55:01.045786 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 06:55:01.046053 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":345,"threads_started":1}
I20260810 06:55:01.048391 25025 heartbeater.cc:347] Connected to a master server at 127.22.11.190:35111
I20260810 06:55:01.048732 25025 heartbeater.cc:464] Registering TS with master...
I20260810 06:55:01.049571 25025 heartbeater.cc:511] Master 127.22.11.190:35111 requested a full tablet report, sending...
I20260810 06:55:01.051543 24707 ts_manager.cc:194] Registered new tserver with Master: 6b1923979c034041b4cc337671053a45 (127.22.11.129:37157)
I20260810 06:55:01.053200 24707 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.129:60761
W20260810 06:55:01.371798 25030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:01.372052 25030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:01.372133 25030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:01.379192 25030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:01.379279 25030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:01.379388 25030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.130
I20260810 06:55:01.388002 25030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.22.11.130
--webserver_port=0
--tserver_master_addrs=127.22.11.190:35111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.130
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:01.389223 25030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:01.390897 25030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:01.402956 25046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:01.402987 25039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:01.403014 25041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:01.403882 25030 server_base.cc:1034] running on GCE node
I20260810 06:55:01.404458 25030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:01.405359 25030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:01.406561 25030 hybrid_clock.cc:648] HybridClock initialized: now 1786344901406511 us; error 44 us; skew 500 ppm
I20260810 06:55:01.406853 25030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:55:01.408903 25030 webserver.cc:460] Webserver started at http://127.22.11.130:34683/ using document root <none> and password file <none>
I20260810 06:55:01.409536 25030 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:01.409636 25030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:01.409924 25030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:01.412590 25030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data/instance:
uuid: "05d45c4039b04d86af707fe54d23cd10"
format_stamp: "Formatted at 2026-08-10 06:55:01 on dist-test-slave-f7th"
server_key: "59967c9995b45d65366651beb47f95e0"
server_key_iv: "dfc174be17fdf0a72cb0afe3d03249c7"
server_key_version: "encryptionkey@0"
I20260810 06:55:01.413373 25030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance:
uuid: "05d45c4039b04d86af707fe54d23cd10"
format_stamp: "Formatted at 2026-08-10 06:55:01 on dist-test-slave-f7th"
server_key: "59967c9995b45d65366651beb47f95e0"
server_key_iv: "dfc174be17fdf0a72cb0afe3d03249c7"
server_key_version: "encryptionkey@0"
I20260810 06:55:01.418067 25030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 06:55:01.421329 25055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:01.422525 25030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:55:01.422667 25030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal
uuid: "05d45c4039b04d86af707fe54d23cd10"
format_stamp: "Formatted at 2026-08-10 06:55:01 on dist-test-slave-f7th"
server_key: "59967c9995b45d65366651beb47f95e0"
server_key_iv: "dfc174be17fdf0a72cb0afe3d03249c7"
server_key_version: "encryptionkey@0"
I20260810 06:55:01.422847 25030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:01.447527 25030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:01.448284 25030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:01.448576 25030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:01.449522 25030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:55:01.450878 25030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:55:01.450973 25030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:01.451084 25030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:55:01.451161 25030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:01.482512 25030 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.130:42779
I20260810 06:55:01.482560 25239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.130:42779 every 8 connection(s)
I20260810 06:55:01.483763 25030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/data/info.pb
I20260810 06:55:01.489621 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25030
I20260810 06:55:01.489787 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal/instance
I20260810 06:55:01.490248 22574 external_mini_cluster.cc:1428] Setting key 73bc56b3bf9e774f1c4c7b949e55bfca
I20260810 06:55:01.493255 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.22.11.131:0
--local_ip_for_outbound_sockets=127.22.11.131
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:35111
--builtin_ntp_servers=127.22.11.148:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 06:55:01.496795 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.486290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:39491 (local address 127.22.11.190:35111)
0810 06:55:01.486494 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:01.486500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:01.487890 (+ 1390us) server_negotiation.cc:407] Connection header received
0810 06:55:01.490311 (+ 2421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:01.490317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:01.490402 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:01.490532 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:01.491783 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.492666 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.494261 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.494664 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.494772 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:01.496001 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:01.496005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:01.496009 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:01.496025 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:01.496057 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:01.496068 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:01.496141 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:01.496345 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 06:55:01.496441 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:55:01.497697 25246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.486749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:35111 (local address 127.22.11.130:39491)
0810 06:55:01.487741 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:01.487782 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:01.490020 (+ 2238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:01.490737 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:01.490748 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:01.490860 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:01.491624 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:01.491649 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.492814 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.492820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:01.494125 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:01.494137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.494879 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.494887 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:01.495097 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:01.495714 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:55:01.495717 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:01.495752 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:55:01.495828 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:01.496152 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:01.496163 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:01.496171 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:55:01.496322 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 06:55:01.496714 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":277,"threads_started":1}
I20260810 06:55:01.498984 25242 heartbeater.cc:347] Connected to a master server at 127.22.11.190:35111
I20260810 06:55:01.499260 25242 heartbeater.cc:464] Registering TS with master...
I20260810 06:55:01.499933 25242 heartbeater.cc:511] Master 127.22.11.190:35111 requested a full tablet report, sending...
I20260810 06:55:01.501220 24707 ts_manager.cc:194] Registered new tserver with Master: 05d45c4039b04d86af707fe54d23cd10 (127.22.11.130:42779)
I20260810 06:55:01.501997 24707 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.130:39491
W20260810 06:55:01.819996 25247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:01.820242 25247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:01.820369 25247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:01.827533 25247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:01.827623 25247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:01.827754 25247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.22.11.131
I20260810 06:55:01.836499 25247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.22.11.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.22.11.131
--webserver_port=0
--tserver_master_addrs=127.22.11.190:35111
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.22.11.131
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:01.837700 25247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:01.839462 25247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:01.851995 25254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:01.852075 25261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:01.852034 25256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:01.852985 25247 server_base.cc:1034] running on GCE node
I20260810 06:55:01.853566 25247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:01.854457 25247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:01.855682 25247 hybrid_clock.cc:648] HybridClock initialized: now 1786344901855626 us; error 51 us; skew 500 ppm
I20260810 06:55:01.856240 25247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 06:55:01.858440 25247 webserver.cc:460] Webserver started at http://127.22.11.131:36329/ using document root <none> and password file <none>
I20260810 06:55:01.859088 25247 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:01.859193 25247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:01.859484 25247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:01.862294 25247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data/instance:
uuid: "a567b74a294f43f985aad62eb9a10cda"
format_stamp: "Formatted at 2026-08-10 06:55:01 on dist-test-slave-f7th"
server_key: "d95ec3bb5711d9d6c743a36df349a3a6"
server_key_iv: "300d050ba710c5c82dbb9c722c16367c"
server_key_version: "encryptionkey@0"
I20260810 06:55:01.863095 25247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance:
uuid: "a567b74a294f43f985aad62eb9a10cda"
format_stamp: "Formatted at 2026-08-10 06:55:01 on dist-test-slave-f7th"
server_key: "d95ec3bb5711d9d6c743a36df349a3a6"
server_key_iv: "300d050ba710c5c82dbb9c722c16367c"
server_key_version: "encryptionkey@0"
I20260810 06:55:01.868038 25247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:55:01.871418 25270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:01.872676 25247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:55:01.872810 25247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal
uuid: "a567b74a294f43f985aad62eb9a10cda"
format_stamp: "Formatted at 2026-08-10 06:55:01 on dist-test-slave-f7th"
server_key: "d95ec3bb5711d9d6c743a36df349a3a6"
server_key_iv: "300d050ba710c5c82dbb9c722c16367c"
server_key_version: "encryptionkey@0"
I20260810 06:55:01.872980 25247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:01.913404 25247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:01.914168 25247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:01.914417 25247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:01.915314 25247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 06:55:01.916736 25247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 06:55:01.916828 25247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:01.916905 25247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 06:55:01.916961 25247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:01.946095 25247 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.131:33653
I20260810 06:55:01.946146 25458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.131:33653 every 8 connection(s)
I20260810 06:55:01.947327 25247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/data/info.pb
I20260810 06:55:01.953049 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25247
I20260810 06:55:01.953200 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal/instance
I20260810 06:55:01.953634 22574 external_mini_cluster.cc:1428] Setting key f374e9917d3bf3fced698947d963898c
I20260810 06:55:01.959517 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.949750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.131:38583 (local address 127.22.11.190:35111)
0810 06:55:01.949925 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:01.949930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:01.951239 (+ 1309us) server_negotiation.cc:407] Connection header received
0810 06:55:01.953017 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:01.953022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:01.953106 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:01.953201 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:01.954378 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.955371 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.957007 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.957407 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.957525 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:01.958739 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:01.958745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:01.958749 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:01.958766 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:01.958798 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:01.958809 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:01.958883 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:01.959096 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 06:55:01.959191 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:55:01.960410 25464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.950241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:35111 (local address 127.22.11.131:38583)
0810 06:55:01.951091 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:01.951133 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:01.952762 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:01.953438 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:01.953446 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:01.953537 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:01.954163 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:01.954187 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.955517 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.955521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:01.956863 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:01.956875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.957619 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.957628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:01.957826 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:01.958445 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 06:55:01.958447 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:01.958485 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 06:55:01.958562 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:01.958893 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:01.958905 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:01.958913 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 06:55:01.959071 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 06:55:01.959426 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":253,"threads_started":1}
I20260810 06:55:01.961539 25460 heartbeater.cc:347] Connected to a master server at 127.22.11.190:35111
I20260810 06:55:01.961773 25460 heartbeater.cc:464] Registering TS with master...
I20260810 06:55:01.962422 25460 heartbeater.cc:511] Master 127.22.11.190:35111 requested a full tablet report, sending...
I20260810 06:55:01.963627 24707 ts_manager.cc:194] Registered new tserver with Master: a567b74a294f43f985aad62eb9a10cda (127.22.11.131:33653)
I20260810 06:55:01.964347 24707 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.22.11.131:38583
I20260810 06:55:01.968245 22574 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 06:55:01.977728 25028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.972048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.22.11.190:35111)
0810 06:55:01.972218 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:01.972224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:01.972662 (+ 438us) server_negotiation.cc:407] Connection header received
0810 06:55:01.972833 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:01.972837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:01.972905 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:01.972986 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:01.973464 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.974112 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.975406 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:01.975836 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:01.975941 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:01.977083 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:01.977089 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:01.977094 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:01.977114 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:01.977151 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:01.977167 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:01.977264 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:01.977379 (+ 115us) server_negotiation.cc:299] Negotiation successful
0810 06:55:01.977461 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 06:55:01.981050 24707 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 06:55:01.987581 24707 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 06:55:02.009080 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.998504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39854)
0810 06:55:02.000255 (+ 1751us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.000286 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:02.000452 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:02.001506 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:02.001513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:02.001544 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:55:02.001965 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.001978 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.003926 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.003931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.006512 (+ 2581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.006528 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.008261 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.008265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.008403 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.008436 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:55:02.008555 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 06:55:02.008747 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1574,"mutex_wait_us":1,"thread_start_us":2019,"threads_started":1}
I20260810 06:55:02.009138 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.999076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.22.11.129:37157)
0810 06:55:01.999839 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:01.999844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.000391 (+ 547us) server_negotiation.cc:407] Connection header received
0810 06:55:02.000765 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.000773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.001015 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.001170 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:55:02.002196 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.003758 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.006729 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.008096 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.008236 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.008476 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.008646 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.008811 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":523,"threads_started":1}
I20260810 06:55:02.010263 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.998580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:33653 (local address 127.0.0.1:53116)
0810 06:55:02.000842 (+ 2262us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.000858 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:02.000965 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:02.001881 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:02.001886 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:02.001914 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:55:02.002276 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.002288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.004098 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.004108 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.006330 (+ 2222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.006342 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.009501 (+ 3159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.009508 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.009687 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.009720 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 06:55:02.009810 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 06:55:02.009942 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2135,"mutex_wait_us":93,"thread_start_us":1389,"threads_started":1}
I20260810 06:55:02.010389 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:01.999002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.22.11.131:33653)
0810 06:55:01.999640 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:01.999649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.001131 (+ 1482us) server_negotiation.cc:407] Connection header received
0810 06:55:02.001233 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.001241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.001442 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.001626 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:55:02.002473 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.003948 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.006513 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.007897 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.009116 (+ 1219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.009382 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.009882 (+ 500us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.010037 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":333,"threads_started":1}
I20260810 06:55:02.011531 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.000793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.130:42779 (local address 127.0.0.1:39268)
0810 06:55:02.001250 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.001265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:02.001372 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:02.002400 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:02.002405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:02.002429 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:55:02.002735 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.002744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.004149 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.004153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.006543 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.006554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.010826 (+ 4272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.010832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.010991 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.011023 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 06:55:02.011105 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:55:02.011218 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":7,"thread_start_us":666,"threads_started":1}
I20260810 06:55:02.011822 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.001268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.22.11.130:42779)
0810 06:55:02.001824 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:02.001833 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.001860 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:55:02.001952 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.001958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.002186 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.002892 (+ 706us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:55:02.003074 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.003999 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.008568 (+ 4569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.010654 (+ 2086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.010770 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.010983 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.011188 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.011347 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":1040,"threads_started":1}
I20260810 06:55:02.013113 24909 tablet_service.cc:1467] Processing CreateTablet for tablet b0d803138f124a82908fddaed3ec6d2a (DEFAULT_TABLE table=test-table [id=f10bfdd1525b4d14b9f813c888091c52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:55:02.013895 25348 tablet_service.cc:1467] Processing CreateTablet for tablet b0d803138f124a82908fddaed3ec6d2a (DEFAULT_TABLE table=test-table [id=f10bfdd1525b4d14b9f813c888091c52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:55:02.014612 24909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d803138f124a82908fddaed3ec6d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:02.014650 25133 tablet_service.cc:1467] Processing CreateTablet for tablet b0d803138f124a82908fddaed3ec6d2a (DEFAULT_TABLE table=test-table [id=f10bfdd1525b4d14b9f813c888091c52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 06:55:02.015295 25348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d803138f124a82908fddaed3ec6d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:02.015647 25133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0d803138f124a82908fddaed3ec6d2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:02.026455 25496 tablet_bootstrap.cc:492] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Bootstrap starting.
I20260810 06:55:02.029096 25497 tablet_bootstrap.cc:492] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: Bootstrap starting.
I20260810 06:55:02.030299 25498 tablet_bootstrap.cc:492] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Bootstrap starting.
I20260810 06:55:02.030671 25496 tablet_bootstrap.cc:654] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:02.032001 25496 log.cc:824] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:02.033607 25498 tablet_bootstrap.cc:654] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:02.033783 25497 tablet_bootstrap.cc:654] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:02.034696 25498 log.cc:824] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:02.035110 25497 log.cc:824] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:02.049649 25497 tablet_bootstrap.cc:492] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: No bootstrap required, opened a new log
I20260810 06:55:02.049645 25498 tablet_bootstrap.cc:492] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: No bootstrap required, opened a new log
I20260810 06:55:02.049945 25497 ts_tablet_manager.cc:1397] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.003s
I20260810 06:55:02.050081 25498 ts_tablet_manager.cc:1397] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.013s
I20260810 06:55:02.050104 25496 tablet_bootstrap.cc:492] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: No bootstrap required, opened a new log
I20260810 06:55:02.050344 25496 ts_tablet_manager.cc:1397] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.008s
I20260810 06:55:02.056131 25025 heartbeater.cc:503] Master 127.22.11.190:35111 was elected leader, sending a full tablet report...
I20260810 06:55:02.057904 25498 raft_consensus.cc:348] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.058359 25498 raft_consensus.cc:374] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:02.058452 25498 raft_consensus.cc:729] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a567b74a294f43f985aad62eb9a10cda, State: Initialized, Role: FOLLOWER
I20260810 06:55:02.059046 25498 consensus_queue.cc:260] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.060271 25496 raft_consensus.cc:348] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.060271 25497 raft_consensus.cc:348] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.060690 25497 raft_consensus.cc:374] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:02.060690 25496 raft_consensus.cc:374] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:02.060789 25497 raft_consensus.cc:729] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1923979c034041b4cc337671053a45, State: Initialized, Role: FOLLOWER
I20260810 06:55:02.060791 25496 raft_consensus.cc:729] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05d45c4039b04d86af707fe54d23cd10, State: Initialized, Role: FOLLOWER
I20260810 06:55:02.061340 25497 consensus_queue.cc:260] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.061419 25496 consensus_queue.cc:260] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.062927 25498 ts_tablet_manager.cc:1428] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260810 06:55:02.063469 25460 heartbeater.cc:503] Master 127.22.11.190:35111 was elected leader, sending a full tablet report...
I20260810 06:55:02.065514 25242 heartbeater.cc:503] Master 127.22.11.190:35111 was elected leader, sending a full tablet report...
I20260810 06:55:02.065653 25497 ts_tablet_manager.cc:1428] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260810 06:55:02.065757 25496 ts_tablet_manager.cc:1428] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Time spent starting tablet: real 0.015s user 0.013s sys 0.003s
I20260810 06:55:02.129864 25507 raft_consensus.cc:482] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 06:55:02.130206 25507 raft_consensus.cc:504] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.131945 25507 leader_election.cc:288] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b1923979c034041b4cc337671053a45 (127.22.11.129:37157), a567b74a294f43f985aad62eb9a10cda (127.22.11.131:33653)
I20260810 06:55:02.138804 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.132320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:52011 (local address 127.22.11.129:37157)
0810 06:55:02.132523 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:02.132530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.133109 (+ 579us) server_negotiation.cc:407] Connection header received
0810 06:55:02.133341 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.133347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.133444 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.133555 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:55:02.134162 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.134993 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.137195 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.137943 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.138085 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.138157 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.138442 (+ 285us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.138565 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:55:02.138988 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.132398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.22.11.130:52159 (local address 127.22.11.131:33653)
0810 06:55:02.132652 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:02.132658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.133701 (+ 1043us) server_negotiation.cc:407] Connection header received
0810 06:55:02.133852 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.133856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.133931 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.134032 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 06:55:02.134601 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.135404 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.137391 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.138161 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.138307 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.138376 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.138647 (+ 271us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.138763 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 06:55:02.139060 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.132916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.131:33653 (local address 127.22.11.130:52159)
0810 06:55:02.133584 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.133609 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:02.133740 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:02.134050 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:02.134054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:02.134073 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:55:02.134448 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.134460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.135609 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.135614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.137211 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.137224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.138345 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.138352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.138496 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.138519 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 06:55:02.138601 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 06:55:02.138714 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":99,"thread_start_us":338,"threads_started":1}
I20260810 06:55:02.139283 25508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.132248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.22.11.130:52011)
0810 06:55:02.132972 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.133002 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:02.133175 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:02.133588 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:02.133591 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:02.133616 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 06:55:02.134008 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.134022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.135120 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.135126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.137001 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:02.137014 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.138126 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.138133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:02.138281 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.138306 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 06:55:02.138392 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 06:55:02.138517 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":88,"thread_start_us":315,"threads_started":1}
I20260810 06:55:02.140033 25382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d803138f124a82908fddaed3ec6d2a" candidate_uuid: "05d45c4039b04d86af707fe54d23cd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a567b74a294f43f985aad62eb9a10cda" is_pre_election: true
I20260810 06:55:02.140143 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d803138f124a82908fddaed3ec6d2a" candidate_uuid: "05d45c4039b04d86af707fe54d23cd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1923979c034041b4cc337671053a45" is_pre_election: true
I20260810 06:55:02.140524 25382 raft_consensus.cc:2454] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d45c4039b04d86af707fe54d23cd10 in term 0.
I20260810 06:55:02.140671 24941 raft_consensus.cc:2454] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05d45c4039b04d86af707fe54d23cd10 in term 0.
I20260810 06:55:02.141209 25058 leader_election.cc:302] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d45c4039b04d86af707fe54d23cd10, a567b74a294f43f985aad62eb9a10cda; no voters:
I20260810 06:55:02.141575 25507 raft_consensus.cc:2781] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 06:55:02.141731 25507 raft_consensus.cc:482] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 06:55:02.141846 25507 raft_consensus.cc:3037] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:02.143987 25507 raft_consensus.cc:504] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.144518 25507 leader_election.cc:288] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [CANDIDATE]: Term 1 election: Requested vote from peers 6b1923979c034041b4cc337671053a45 (127.22.11.129:37157), a567b74a294f43f985aad62eb9a10cda (127.22.11.131:33653)
I20260810 06:55:02.145015 25382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d803138f124a82908fddaed3ec6d2a" candidate_uuid: "05d45c4039b04d86af707fe54d23cd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a567b74a294f43f985aad62eb9a10cda"
I20260810 06:55:02.145015 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0d803138f124a82908fddaed3ec6d2a" candidate_uuid: "05d45c4039b04d86af707fe54d23cd10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1923979c034041b4cc337671053a45"
I20260810 06:55:02.145233 24941 raft_consensus.cc:3037] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:02.145233 25382 raft_consensus.cc:3037] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:02.148411 24941 raft_consensus.cc:2454] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d45c4039b04d86af707fe54d23cd10 in term 1.
I20260810 06:55:02.148412 25382 raft_consensus.cc:2454] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05d45c4039b04d86af707fe54d23cd10 in term 1.
I20260810 06:55:02.148823 25059 leader_election.cc:302] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05d45c4039b04d86af707fe54d23cd10, 6b1923979c034041b4cc337671053a45; no voters:
I20260810 06:55:02.149122 25507 raft_consensus.cc:2781] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:02.149606 25507 raft_consensus.cc:686] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 1 LEADER]: Becoming Leader. State: Replica: 05d45c4039b04d86af707fe54d23cd10, State: Running, Role: LEADER
I20260810 06:55:02.150058 25507 consensus_queue.cc:237] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } }
I20260810 06:55:02.154603 24706 catalog_manager.cc:5495] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 05d45c4039b04d86af707fe54d23cd10 (127.22.11.130). New cstate: current_term: 1 leader_uuid: "05d45c4039b04d86af707fe54d23cd10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05d45c4039b04d86af707fe54d23cd10" member_type: VOTER last_known_addr { host: "127.22.11.130" port: 42779 } health_report { overall_health: HEALTHY } } }
W20260810 06:55:02.198839 25463 tablet.cc:2369] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:55:02.201100 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.195774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.22.11.130:42779)
0810 06:55:02.196002 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:02.196007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.196025 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:55:02.196191 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.196195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.196285 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.196404 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:02.196717 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.197453 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.198920 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.199556 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.199685 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.200104 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:02.200578 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:02.200731 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.200776 (+ 45us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.200841 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 06:55:02.209283 25382 raft_consensus.cc:1261] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 1 FOLLOWER]: Refusing update from remote peer 05d45c4039b04d86af707fe54d23cd10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:55:02.209283 24941 raft_consensus.cc:1261] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 1 FOLLOWER]: Refusing update from remote peer 05d45c4039b04d86af707fe54d23cd10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:55:02.210276 25511 consensus_queue.cc:1035] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b1923979c034041b4cc337671053a45" member_type: VOTER last_known_addr { host: "127.22.11.129" port: 37157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:55:02.210636 25507 consensus_queue.cc:1035] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a567b74a294f43f985aad62eb9a10cda" member_type: VOTER last_known_addr { host: "127.22.11.131" port: 33653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:55:02.220754 25516 mvcc.cc:205] Tried to move back new op lower bound from 7316868719438180352 to 7316868719210610688. Current Snapshot: MvccSnapshot[applied={T|T < 7316868719438180352}]
I20260810 06:55:02.235743 24707 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:55:02.235984 24707 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 06:55:02.239419 24707 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd7914f0fd9546a3b263210f6d9f4775: Sending DeleteTablet for 3 replicas of tablet b0d803138f124a82908fddaed3ec6d2a
I20260810 06:55:02.240396 24909 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d803138f124a82908fddaed3ec6d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:55:02 UTC) from {username='slave'} at 127.0.0.1:39854
I20260810 06:55:02.240799 25348 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d803138f124a82908fddaed3ec6d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:55:02 UTC) from {username='slave'} at 127.0.0.1:53116
I20260810 06:55:02.241098 25133 tablet_service.cc:1514] Processing DeleteTablet for tablet b0d803138f124a82908fddaed3ec6d2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 06:55:02 UTC) from {username='slave'} at 127.0.0.1:39268
I20260810 06:55:02.242091 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24806
I20260810 06:55:02.259467 25536 tablet_replica.cc:330] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: stopping tablet replica
I20260810 06:55:02.261701 25543 tablet_replica.cc:330] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: stopping tablet replica
I20260810 06:55:02.261905 25536 raft_consensus.cc:2229] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:55:02.262467 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.243445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.22.11.129:37157)
0810 06:55:02.243619 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:02.243625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.252960 (+ 9335us) server_negotiation.cc:407] Connection header received
0810 06:55:02.253405 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.253412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.253515 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.253591 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:02.254280 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.255493 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.257438 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.258118 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.258218 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.260002 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:02.260014 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:02.260031 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:02.260833 (+ 802us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:02.260905 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:02.260930 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:02.261023 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.261953 (+ 930us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.262069 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:55:02.262871 25538 tablet_replica.cc:330] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: stopping tablet replica
I20260810 06:55:02.262939 25536 raft_consensus.cc:2258] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:55:02.263234 25538 raft_consensus.cc:2229] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 06:55:02.263692 25538 raft_consensus.cc:2258] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:55:02.264245 25014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:02.264479 25543 raft_consensus.cc:2229] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 1 LEADER]: Raft consensus shutting down.
I20260810 06:55:02.265142 25543 raft_consensus.cc:2258] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 06:55:02.265846 25538 ts_tablet_manager.cc:1905] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:55:02.267683 25543 ts_tablet_manager.cc:1905] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:55:02.272938 25538 ts_tablet_manager.cc:1918] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:55:02.273100 25538 log.cc:1196] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-2/wal/wals/b0d803138f124a82908fddaed3ec6d2a
I20260810 06:55:02.273645 25538 ts_tablet_manager.cc:1939] T b0d803138f124a82908fddaed3ec6d2a P a567b74a294f43f985aad62eb9a10cda: Deleting consensus metadata
I20260810 06:55:02.275483 24677 catalog_manager.cc:4841] TS a567b74a294f43f985aad62eb9a10cda (127.22.11.131:33653): tablet b0d803138f124a82908fddaed3ec6d2a (table test-table [id=f10bfdd1525b4d14b9f813c888091c52]) successfully deleted
I20260810 06:55:02.276276 25543 ts_tablet_manager.cc:1918] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 06:55:02.276561 25543 log.cc:1196] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Deleting WAL directory at /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786344580063516-22574-0/minicluster-data/ts-1/wal/wals/b0d803138f124a82908fddaed3ec6d2a
I20260810 06:55:02.277155 25543 ts_tablet_manager.cc:1939] T b0d803138f124a82908fddaed3ec6d2a P 05d45c4039b04d86af707fe54d23cd10: Deleting consensus metadata
I20260810 06:55:02.278883 24677 catalog_manager.cc:4841] TS 05d45c4039b04d86af707fe54d23cd10 (127.22.11.130:42779): tablet b0d803138f124a82908fddaed3ec6d2a (table test-table [id=f10bfdd1525b4d14b9f813c888091c52]) successfully deleted
I20260810 06:55:02.382311 25536 ts_tablet_manager.cc:1905] T b0d803138f124a82908fddaed3ec6d2a P 6b1923979c034041b4cc337671053a45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 06:55:02.383395 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24806
I20260810 06:55:02.405289 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25030
W20260810 06:55:02.405318 24681 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.22.11.129:37157 (error 108)
I20260810 06:55:02.407503 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.406951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39868)
0810 06:55:02.407204 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.407332 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260810 06:55:02.407755 24681 catalog_manager.cc:4584] TS 6b1923979c034041b4cc337671053a45 (127.22.11.129:37157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b0d803138f124a82908fddaed3ec6d2a: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
I20260810 06:55:02.412755 25495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.406372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.22.11.130:42779)
0810 06:55:02.406573 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:02.406579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.406601 (+ 22us) server_negotiation.cc:407] Connection header received
0810 06:55:02.406699 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.406703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.406806 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.406920 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:02.407387 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.408292 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.409603 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.410054 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.410158 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.411239 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:02.411248 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:02.411272 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:02.411893 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:02.411945 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:02.411971 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:02.412064 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.412216 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.412319 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:55:02.413524 25228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:02.461462 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.461003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39874)
0810 06:55:02.461201 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.461289 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:55:02.462481 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.462160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39876)
0810 06:55:02.462283 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.462356 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260810 06:55:02.523068 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.522445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39878)
0810 06:55:02.522735 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.522853 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 06:55:02.524107 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.523741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39888)
0810 06:55:02.523887 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.523975 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 06:55:02.525748 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25030
I20260810 06:55:02.547652 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25247
I20260810 06:55:02.554232 25486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.548372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53132 (local address 127.22.11.131:33653)
0810 06:55:02.548567 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:02.548573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:02.548593 (+ 20us) server_negotiation.cc:407] Connection header received
0810 06:55:02.548789 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:02.548793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:02.548876 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:02.548964 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:02.549394 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.550121 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.551327 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:02.551713 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:02.551823 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:02.552933 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:02.552943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:02.552959 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:02.553532 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:02.553588 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:02.553608 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:02.553709 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:02.553841 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 06:55:02.553949 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:55:02.554853 25451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:02.605357 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.604862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39902)
0810 06:55:02.605085 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.605180 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 06:55:02.606405 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:02.606008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.129:37157 (local address 127.0.0.1:39914)
0810 06:55:02.606216 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:02.606297 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.129:37157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 06:55:02.642392 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25247
I20260810 06:55:02.666884 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 24651
I20260810 06:55:02.667975 24763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:02.774560 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 24651
2026-08-10T06:55:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2796 ms)
[----------] 6 tests from EncryptionPolicyTest (21583 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:02 dist-test-slave-f7th krb5kdc[25561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:02 dist-test-slave-f7th krb5kdc[25561](info): set up 2 sockets
Aug 10 06:55:02 dist-test-slave-f7th krb5kdc[25561](info): commencing operation
krb5kdc: starting...
W20260810 06:55:04.046654 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:04 dist-test-slave-f7th krb5kdc[25561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:04.341614 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:04.671319 25578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:04.671608 25578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:04.671741 25578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:04.678719 25578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:04.678835 25578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:04.678926 25578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:04.679011 25578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:04.679075 25578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:04.688553 25578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44611
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:04.689774 25578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:04.691047 25578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:04.702353 25591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:04.702353 25590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:04.702389 25593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:04.703045 25578 server_base.cc:1034] running on GCE node
I20260810 06:55:04.703715 25578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:04.705020 25578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:04.706221 25578 hybrid_clock.cc:648] HybridClock initialized: now 1786344904706181 us; error 31 us; skew 500 ppm
I20260810 06:55:04.706552 25578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:04 dist-test-slave-f7th krb5kdc[25561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344904, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:04.709547 25578 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:04.710956 25578 webserver.cc:460] Webserver started at http://127.0.0.1:45885/ using document root <none> and password file <none>
I20260810 06:55:04.711570 25578 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:04.711683 25578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:04.711982 25578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:04.714913 25578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "26393821258644c8a0c3258990b37cd8"
format_stamp: "Formatted at 2026-08-10 06:55:04 on dist-test-slave-f7th"
server_key: "b38520fed500625859d0d750e192408b"
server_key_iv: "09ed016f528dc01febee6f7a0d4f0eba"
server_key_version: "encryptionkey@0"
I20260810 06:55:04.715651 25578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "26393821258644c8a0c3258990b37cd8"
format_stamp: "Formatted at 2026-08-10 06:55:04 on dist-test-slave-f7th"
server_key: "b38520fed500625859d0d750e192408b"
server_key_iv: "09ed016f528dc01febee6f7a0d4f0eba"
server_key_version: "encryptionkey@0"
I20260810 06:55:04.720428 25578 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 06:55:04.723624 25610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:04.724860 25578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 06:55:04.725020 25578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "26393821258644c8a0c3258990b37cd8"
format_stamp: "Formatted at 2026-08-10 06:55:04 on dist-test-slave-f7th"
server_key: "b38520fed500625859d0d750e192408b"
server_key_iv: "09ed016f528dc01febee6f7a0d4f0eba"
server_key_version: "encryptionkey@0"
I20260810 06:55:04.725184 25578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:04.752885 25578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:04.756657 25578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:04.756924 25578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:04.774127 25578 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44611
I20260810 06:55:04.774147 25696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44611 every 8 connection(s)
I20260810 06:55:04.775403 25578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:04.779129 25698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:04.779146 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25578
I20260810 06:55:04.779340 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:04.780162 22574 external_mini_cluster.cc:1428] Setting key 99af0ad4ff2a487273fafd7acbb86aa1
I20260810 06:55:04.792595 25698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8: Bootstrap starting.
Aug 10 06:55:04 dist-test-slave-f7th krb5kdc[25561](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:55:04.796329 25698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:04.797442 25698 log.cc:824] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:04.800710 25698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8: No bootstrap required, opened a new log
I20260810 06:55:04.804401 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:04.782048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60945 (local address 127.0.0.1:44611)
0810 06:55:04.782805 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:04.782815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:04.782853 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:55:04.785154 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:04.785176 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:04.785550 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:04.785867 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:04.787615 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:04.788926 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:04.790651 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:04.791312 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:04.791438 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:04.796256 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:04.796264 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:04.796277 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:04.796314 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:04.799830 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:04.800594 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:04.800602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:04.800609 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:04.800890 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:04.801326 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:04.801330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:04.801333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:04.802028 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:04.802334 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:04.802716 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 06:55:04.803070 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":361,"threads_started":1}
I20260810 06:55:04.808295 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:04.809939 25698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26393821258644c8a0c3258990b37cd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260810 06:55:04.810254 25698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:04.810355 25698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26393821258644c8a0c3258990b37cd8, State: Initialized, Role: FOLLOWER
I20260810 06:55:04.810916 25698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26393821258644c8a0c3258990b37cd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260810 06:55:04.811149 25698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:04.811298 25698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:04.811489 25698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:04.813817 25698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26393821258644c8a0c3258990b37cd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260810 06:55:04.814368 25698 leader_election.cc:302] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26393821258644c8a0c3258990b37cd8; no voters:
I20260810 06:55:04.815034 25698 leader_election.cc:288] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:04.815142 25722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:04.816875 25722 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [term 1 LEADER]: Becoming Leader. State: Replica: 26393821258644c8a0c3258990b37cd8, State: Running, Role: LEADER
I20260810 06:55:04.817377 25698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:04.817490 25722 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26393821258644c8a0c3258990b37cd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260810 06:55:04.820310 25723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26393821258644c8a0c3258990b37cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26393821258644c8a0c3258990b37cd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260810 06:55:04.820827 25723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:04.821308 25728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26393821258644c8a0c3258990b37cd8. Latest consensus state: current_term: 1 leader_uuid: "26393821258644c8a0c3258990b37cd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26393821258644c8a0c3258990b37cd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260810 06:55:04.821583 25728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:04.830849 25705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:04.813013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40029 (local address 127.0.0.1:44611)
0810 06:55:04.813194 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:04.813200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:04.814520 (+ 1320us) server_negotiation.cc:407] Connection header received
0810 06:55:04.814697 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:04.814700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:04.814756 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:04.814847 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:04.816524 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:04.817509 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:04.818838 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:04.819422 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:04.819551 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:04.822713 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:04.822720 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:04.822726 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:04.822765 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:04.826494 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:04.827498 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:04.827503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:04.827507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:04.827652 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:04.828218 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:04.828224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:04.828227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:04.828664 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:04.828954 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:04.830273 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0810 06:55:04.830468 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:55:04.833256 25733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:04.838855 25733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:04.847728 25733 catalog_manager.cc:1347] Generated new cluster ID: 9e912c3f610d4ffa9a603a4115cfb198
I20260810 06:55:04.847846 25733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:04.863804 25733 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:04.865674 25733 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:04.881175 25733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26393821258644c8a0c3258990b37cd8: Generated new TSK 0
I20260810 06:55:04.882208 25733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:04.888769 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25578
I20260810 06:55:04.890208 25690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:04.979385 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25578
W20260810 06:55:05.004343 25555 connection.cc:535] client connection to 127.0.0.1:44611 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44611): Connection reset by peer (error 104)
2026-08-10T06:55:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2196 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:05 dist-test-slave-f7th krb5kdc[25752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:05 dist-test-slave-f7th krb5kdc[25752](info): set up 2 sockets
Aug 10 06:55:05 dist-test-slave-f7th krb5kdc[25752](info): commencing operation
krb5kdc: starting...
W20260810 06:55:06.230659 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:06 dist-test-slave-f7th krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:06.509640 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45093
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:06.839274 25787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:06.839555 25787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:06.839687 25787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:06.846704 25787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:06.846812 25787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:06.847096 25787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:06.847242 25787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:06.847314 25787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:06.856976 25787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45093
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:06.858240 25787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:06.859510 25787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:06.870584 25793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:06.870580 25794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:06.870900 25796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:06.871469 25787 server_base.cc:1034] running on GCE node
I20260810 06:55:06.872201 25787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:06.873629 25787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:06.874830 25787 hybrid_clock.cc:648] HybridClock initialized: now 1786344906874775 us; error 45 us; skew 500 ppm
I20260810 06:55:06.875154 25787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:06 dist-test-slave-f7th krb5kdc[25752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344906, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:06.881836 25787 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:06.883244 25787 webserver.cc:460] Webserver started at http://127.0.0.1:46723/ using document root <none> and password file <none>
I20260810 06:55:06.883899 25787 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:06.884016 25787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:06.884292 25787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:06.887233 25787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "81bfa5ee14f446b3bdb9940eef4cdddb"
format_stamp: "Formatted at 2026-08-10 06:55:06 on dist-test-slave-f7th"
server_key: "2a5cb79472a60c7afa471d050e2ea060"
server_key_iv: "cbb3faa5549123785d35711dc98a9874"
server_key_version: "encryptionkey@0"
I20260810 06:55:06.887966 25787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "81bfa5ee14f446b3bdb9940eef4cdddb"
format_stamp: "Formatted at 2026-08-10 06:55:06 on dist-test-slave-f7th"
server_key: "2a5cb79472a60c7afa471d050e2ea060"
server_key_iv: "cbb3faa5549123785d35711dc98a9874"
server_key_version: "encryptionkey@0"
I20260810 06:55:06.892843 25787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 06:55:06.896111 25804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:06.897339 25787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:55:06.897498 25787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "81bfa5ee14f446b3bdb9940eef4cdddb"
format_stamp: "Formatted at 2026-08-10 06:55:06 on dist-test-slave-f7th"
server_key: "2a5cb79472a60c7afa471d050e2ea060"
server_key_iv: "cbb3faa5549123785d35711dc98a9874"
server_key_version: "encryptionkey@0"
I20260810 06:55:06.897670 25787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:06.916271 25787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:06.919951 25787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:06.920222 25787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:06.937788 25787 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45093
I20260810 06:55:06.937825 25895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45093 every 8 connection(s)
I20260810 06:55:06.939054 25787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:06.942651 25896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:06.946053 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25787
I20260810 06:55:06.946240 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:06.946810 22574 external_mini_cluster.cc:1428] Setting key 00769dbe588c2650d06d372f24048a4a
I20260810 06:55:06.954756 25896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb: Bootstrap starting.
I20260810 06:55:06.958237 25896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:06.959340 25896 log.cc:824] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:06.959733 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:06.948499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46433 (local address 127.0.0.1:45093)
0810 06:55:06.949234 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:06.949246 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:06.949284 (+ 38us) server_negotiation.cc:407] Connection header received
0810 06:55:06.950965 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:06.950988 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:06.951386 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:06.951698 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:06.952287 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:06.953638 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:06.955449 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:06.956097 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:06.956226 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:06.957467 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:06.957474 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:06.957486 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:06.957980 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:06.958057 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:06.958081 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:06.958203 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:06.958571 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 06:55:06.958762 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":303,"threads_started":1}
I20260810 06:55:06.962152 25896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb: No bootstrap required, opened a new log
I20260810 06:55:06.963349 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:06.971760 25896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260810 06:55:06.972165 25896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:06.972285 25896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81bfa5ee14f446b3bdb9940eef4cdddb, State: Initialized, Role: FOLLOWER
I20260810 06:55:06.972903 25896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260810 06:55:06.973124 25896 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:06.973280 25896 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:06.973464 25896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:06.974998 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:06.968426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34429 (local address 127.0.0.1:45093)
0810 06:55:06.968591 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:06.968597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:06.969125 (+ 528us) server_negotiation.cc:407] Connection header received
0810 06:55:06.969281 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:06.969284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:06.969338 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:06.969416 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:06.969980 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:06.970733 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:06.972527 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:06.973056 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:06.973176 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:06.974282 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:06.974287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:06.974290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:06.974305 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:06.974360 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:06.974376 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:06.974462 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:06.974640 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 06:55:06.974681 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:55:06.976279 25896 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260810 06:55:06.976787 25896 leader_election.cc:302] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81bfa5ee14f446b3bdb9940eef4cdddb; no voters:
I20260810 06:55:06.977295 25896 leader_election.cc:288] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:06.977399 25916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:06.978039 25916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [term 1 LEADER]: Becoming Leader. State: Replica: 81bfa5ee14f446b3bdb9940eef4cdddb, State: Running, Role: LEADER
I20260810 06:55:06.978583 25916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } }
I20260810 06:55:06.979211 25896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:06.981825 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81bfa5ee14f446b3bdb9940eef4cdddb. Latest consensus state: current_term: 1 leader_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } } }
I20260810 06:55:06.982139 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:06.982889 25917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81bfa5ee14f446b3bdb9940eef4cdddb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45093 } } }
I20260810 06:55:06.983285 25917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:06.983533 25930 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:06.988839 25930 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:06.997375 25930 catalog_manager.cc:1347] Generated new cluster ID: 98898eaf60614648b024cc2c14373d90
I20260810 06:55:06.997478 25930 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:07.020231 25930 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:07.021601 25930 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:07.026434 25930 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81bfa5ee14f446b3bdb9940eef4cdddb: Generated new TSK 0
I20260810 06:55:07.027210 25930 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:07.033962 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25787
W20260810 06:55:07.035105 25890 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46433
W20260810 06:55:07.035784 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25787: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:07.035995 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25787
W20260810 06:55:07.064682 25749 connection.cc:535] client connection to 127.0.0.1:45093 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45093): Connection reset by peer (error 104)
2026-08-10T06:55:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2060 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:07 dist-test-slave-f7th krb5kdc[25954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:07 dist-test-slave-f7th krb5kdc[25954](info): set up 2 sockets
Aug 10 06:55:07 dist-test-slave-f7th krb5kdc[25954](info): commencing operation
krb5kdc: starting...
W20260810 06:55:08.302274 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:08 dist-test-slave-f7th krb5kdc[25954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:08.589856 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:08.919921 25975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:08.920235 25975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:08.920349 25975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:08.927356 25975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:08.927452 25975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:08.927503 25975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:08.927563 25975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:08.927606 25975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:08.937126 25975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:08.938289 25975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:08.939514 25975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:08.950574 25981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:08.950533 25982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:08.950608 25986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:08.951411 25975 server_base.cc:1034] running on GCE node
I20260810 06:55:08.952185 25975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:08.953570 25975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:08.954798 25975 hybrid_clock.cc:648] HybridClock initialized: now 1786344908954751 us; error 34 us; skew 500 ppm
I20260810 06:55:08.955158 25975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:08 dist-test-slave-f7th krb5kdc[25954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344908, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:08.958199 25975 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:08.959549 25975 webserver.cc:460] Webserver started at http://127.0.0.1:39019/ using document root <none> and password file <none>
I20260810 06:55:08.960186 25975 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:08.960304 25975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:08.960601 25975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:08.963541 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "20acfbae27294b8f9621782bf0c3cd14"
format_stamp: "Formatted at 2026-08-10 06:55:08 on dist-test-slave-f7th"
server_key: "bbfe35561d9fce4a7d5d8160cc0f1bf9"
server_key_iv: "3494dfebbf78cf29444fb893402d8623"
server_key_version: "encryptionkey@0"
I20260810 06:55:08.964259 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "20acfbae27294b8f9621782bf0c3cd14"
format_stamp: "Formatted at 2026-08-10 06:55:08 on dist-test-slave-f7th"
server_key: "bbfe35561d9fce4a7d5d8160cc0f1bf9"
server_key_iv: "3494dfebbf78cf29444fb893402d8623"
server_key_version: "encryptionkey@0"
I20260810 06:55:08.969134 25975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:55:08.972283 25997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:08.973568 25975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:55:08.973726 25975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "20acfbae27294b8f9621782bf0c3cd14"
format_stamp: "Formatted at 2026-08-10 06:55:08 on dist-test-slave-f7th"
server_key: "bbfe35561d9fce4a7d5d8160cc0f1bf9"
server_key_iv: "3494dfebbf78cf29444fb893402d8623"
server_key_version: "encryptionkey@0"
I20260810 06:55:08.973894 25975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:08.979089 25975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:08.982693 25975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:08.982944 25975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:08.999411 25975 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44011
I20260810 06:55:08.999464 26078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44011 every 8 connection(s)
I20260810 06:55:09.000692 25975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:09.003991 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 25975
I20260810 06:55:09.004180 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:09.004447 26079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:09.004801 22574 external_mini_cluster.cc:1428] Setting key 91d41f7c37b5e4605777ab4ae62531d3
I20260810 06:55:09.012161 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:09.006478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43289 (local address 127.0.0.1:44011)
0810 06:55:09.007281 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:09.007292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:09.007337 (+ 45us) server_negotiation.cc:407] Connection header received
0810 06:55:09.009076 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:09.009097 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:09.009407 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:09.009701 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:09.010716 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:09.010722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:09.010737 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:09.010751 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:09.010808 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:09.010831 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:09.011011 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:09.011105 (+ 94us) server_negotiation.cc:299] Negotiation successful
0810 06:55:09.011291 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":341,"threads_started":1}
I20260810 06:55:09.015609 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:09.017530 26079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14: Bootstrap starting.
I20260810 06:55:09.021088 26079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:09.022106 26079 log.cc:824] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:09.022940 26085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:09.020516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38775 (local address 127.0.0.1:44011)
0810 06:55:09.020702 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:09.020708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:09.021140 (+ 432us) server_negotiation.cc:407] Connection header received
0810 06:55:09.021279 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:09.021283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:09.021325 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:09.021414 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:09.022232 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:09.022237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:09.022241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:09.022259 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:09.022299 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:09.022314 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:09.022404 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:09.022525 (+ 121us) server_negotiation.cc:299] Negotiation successful
0810 06:55:09.022575 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:55:09.025063 26079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14: No bootstrap required, opened a new log
I20260810 06:55:09.033084 26079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20acfbae27294b8f9621782bf0c3cd14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44011 } }
I20260810 06:55:09.033408 26079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:09.033484 26079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20acfbae27294b8f9621782bf0c3cd14, State: Initialized, Role: FOLLOWER
I20260810 06:55:09.034001 26079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20acfbae27294b8f9621782bf0c3cd14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44011 } }
I20260810 06:55:09.034219 26079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:09.034314 26079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:09.034433 26079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:09.036425 26079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20acfbae27294b8f9621782bf0c3cd14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44011 } }
I20260810 06:55:09.036867 26079 leader_election.cc:302] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20acfbae27294b8f9621782bf0c3cd14; no voters:
I20260810 06:55:09.037366 26079 leader_election.cc:288] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:09.037485 26095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:09.038380 26095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [term 1 LEADER]: Becoming Leader. State: Replica: 20acfbae27294b8f9621782bf0c3cd14, State: Running, Role: LEADER
I20260810 06:55:09.038949 26095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20acfbae27294b8f9621782bf0c3cd14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44011 } }
I20260810 06:55:09.039263 26079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:09.041455 26098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20acfbae27294b8f9621782bf0c3cd14. Latest consensus state: current_term: 1 leader_uuid: "20acfbae27294b8f9621782bf0c3cd14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20acfbae27294b8f9621782bf0c3cd14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44011 } } }
I20260810 06:55:09.041481 26096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20acfbae27294b8f9621782bf0c3cd14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20acfbae27294b8f9621782bf0c3cd14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44011 } } }
I20260810 06:55:09.041926 26098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:09.041926 26096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:09.042876 26107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:09.052265 26107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:09.059984 26107 catalog_manager.cc:1347] Generated new cluster ID: 1c809fad8dd04599bd4de413a1981cb7
I20260810 06:55:09.060082 26107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:09.081633 26107 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:09.082026 26107 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:09.091158 26107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20acfbae27294b8f9621782bf0c3cd14: Generated new TSK 0
I20260810 06:55:09.091857 26107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:09.098757 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25975
W20260810 06:55:09.099714 26073 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43289
W20260810 06:55:09.100359 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 25975: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:09.100483 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 25975
2026-08-10T06:55:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:09 dist-test-slave-f7th krb5kdc[26124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:09 dist-test-slave-f7th krb5kdc[26124](info): set up 2 sockets
Aug 10 06:55:09 dist-test-slave-f7th krb5kdc[26124](info): commencing operation
krb5kdc: starting...
W20260810 06:55:10.355132 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:10 dist-test-slave-f7th krb5kdc[26124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:10.639816 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36315
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:10.985527 26145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:10.985785 26145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:10.985924 26145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:10.992877 26145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:10.992972 26145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:10.993046 26145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:10.993126 26145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:10.993184 26145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:11.002468 26145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:11.003624 26145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:11.004921 26145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:11.015957 26153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:11.015901 26156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:11.015997 26154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:11.017052 26145 server_base.cc:1034] running on GCE node
I20260810 06:55:11.017835 26145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:11.019107 26145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:11.020304 26145 hybrid_clock.cc:648] HybridClock initialized: now 1786344911020262 us; error 28 us; skew 500 ppm
I20260810 06:55:11.020648 26145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:11 dist-test-slave-f7th krb5kdc[26124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344911, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:11.023612 26145 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:11.025060 26145 webserver.cc:460] Webserver started at http://127.0.0.1:44237/ using document root <none> and password file <none>
I20260810 06:55:11.025687 26145 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:11.025803 26145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:11.026087 26145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:11.028998 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "aad77a18f99943288aaba905f7ccf746"
format_stamp: "Formatted at 2026-08-10 06:55:11 on dist-test-slave-f7th"
server_key: "ccd47f2c302cdad34e6bd25f83339e57"
server_key_iv: "a1581782a75eb9325ab529955f018f5d"
server_key_version: "encryptionkey@0"
I20260810 06:55:11.029728 26145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "aad77a18f99943288aaba905f7ccf746"
format_stamp: "Formatted at 2026-08-10 06:55:11 on dist-test-slave-f7th"
server_key: "ccd47f2c302cdad34e6bd25f83339e57"
server_key_iv: "a1581782a75eb9325ab529955f018f5d"
server_key_version: "encryptionkey@0"
I20260810 06:55:11.034461 26145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:55:11.037650 26168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:11.038880 26145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:55:11.039023 26145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "aad77a18f99943288aaba905f7ccf746"
format_stamp: "Formatted at 2026-08-10 06:55:11 on dist-test-slave-f7th"
server_key: "ccd47f2c302cdad34e6bd25f83339e57"
server_key_iv: "a1581782a75eb9325ab529955f018f5d"
server_key_version: "encryptionkey@0"
I20260810 06:55:11.039203 26145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:11.058739 26145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:11.062870 26145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:11.063123 26145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:11.079121 26145 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36315
I20260810 06:55:11.079144 26263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36315 every 8 connection(s)
I20260810 06:55:11.080371 26145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:11.084086 26265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:11.088133 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26145
I20260810 06:55:11.088303 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:11.088768 22574 external_mini_cluster.cc:1428] Setting key e6fe55061a06f0f96441f875a919b47d
I20260810 06:55:11.094555 26265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746: Bootstrap starting.
I20260810 06:55:11.097995 26265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:11.099129 26265 log.cc:824] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:11.102285 26265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746: No bootstrap required, opened a new log
Aug 10 06:55:11 dist-test-slave-f7th krb5kdc[26124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:55:11.111516 26265 raft_consensus.cc:348] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad77a18f99943288aaba905f7ccf746" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36315 } }
I20260810 06:55:11.111877 26265 raft_consensus.cc:374] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:11.111904 26268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:11.090330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33563 (local address 127.0.0.1:36315)
0810 06:55:11.091080 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:11.091090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:11.091147 (+ 57us) server_negotiation.cc:407] Connection header received
0810 06:55:11.093119 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:11.093141 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:11.093484 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:11.093798 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:11.095530 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:11.096881 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:11.098581 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:11.099218 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:11.099361 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:11.104199 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:11.104210 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:11.104223 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:11.104267 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:11.107714 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:11.108534 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:11.108540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:11.108546 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:11.108761 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:11.109187 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:11.109190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:11.109192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:11.109758 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:11.109978 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:11.110515 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 06:55:11.110860 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":364,"threads_started":1}
I20260810 06:55:11.111987 26265 raft_consensus.cc:729] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aad77a18f99943288aaba905f7ccf746, State: Initialized, Role: FOLLOWER
I20260810 06:55:11.112696 26265 consensus_queue.cc:260] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad77a18f99943288aaba905f7ccf746" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36315 } }
I20260810 06:55:11.112984 26265 raft_consensus.cc:388] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:11.113131 26265 raft_consensus.cc:482] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:11.113314 26265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:11.115645 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:11.115973 26265 raft_consensus.cc:504] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad77a18f99943288aaba905f7ccf746" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36315 } }
I20260810 06:55:11.116423 26265 leader_election.cc:302] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aad77a18f99943288aaba905f7ccf746; no voters:
I20260810 06:55:11.117036 26265 leader_election.cc:288] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:11.117182 26270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:11.119012 26265 sys_catalog.cc:564] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:11.124068 26270 raft_consensus.cc:686] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [term 1 LEADER]: Becoming Leader. State: Replica: aad77a18f99943288aaba905f7ccf746, State: Running, Role: LEADER
I20260810 06:55:11.124758 26270 consensus_queue.cc:237] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad77a18f99943288aaba905f7ccf746" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36315 } }
I20260810 06:55:11.127766 26273 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aad77a18f99943288aaba905f7ccf746" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad77a18f99943288aaba905f7ccf746" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36315 } } }
I20260810 06:55:11.128645 26273 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:11.129289 26276 sys_catalog.cc:455] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aad77a18f99943288aaba905f7ccf746. Latest consensus state: current_term: 1 leader_uuid: "aad77a18f99943288aaba905f7ccf746" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aad77a18f99943288aaba905f7ccf746" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36315 } } }
I20260810 06:55:11.129666 26276 sys_catalog.cc:458] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:11.132228 26300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:11.140704 26300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:11.148277 26268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:11.127207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54269 (local address 127.0.0.1:36315)
0810 06:55:11.127916 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:11.127922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:11.128184 (+ 262us) server_negotiation.cc:407] Connection header received
0810 06:55:11.128271 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:11.128277 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:11.128357 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:11.128448 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:11.131548 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:11.132849 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:11.135037 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:11.135761 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:11.135864 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:11.139512 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:11.139520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:11.139526 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:11.139567 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:11.143812 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:11.144799 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:11.144805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:11.144819 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:11.144999 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:11.145517 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:11.145526 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:11.145530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:11.145964 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:11.146225 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:11.147553 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0810 06:55:11.147866 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588}
I20260810 06:55:11.150404 26300 catalog_manager.cc:1347] Generated new cluster ID: cc73498b99034104810b74d583488210
I20260810 06:55:11.150518 26300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:11.179916 26300 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:11.181229 26300 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:11.191457 26300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aad77a18f99943288aaba905f7ccf746: Generated new TSK 0
I20260810 06:55:11.192198 26300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:11.201627 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26145
I20260810 06:55:11.202962 26258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:11.295580 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26145
2026-08-10T06:55:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2194 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:11 dist-test-slave-f7th krb5kdc[26318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:11 dist-test-slave-f7th krb5kdc[26318](info): set up 2 sockets
Aug 10 06:55:11 dist-test-slave-f7th krb5kdc[26318](info): commencing operation
krb5kdc: starting...
W20260810 06:55:12.540300 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:12 dist-test-slave-f7th krb5kdc[26318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:12.824849 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:13.155839 26338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:13.156127 26338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:13.156235 26338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:13.163273 26338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:13.163379 26338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:13.163431 26338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:13.163491 26338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:13.163535 26338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:13.172983 26338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:13.174126 26338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:13.175375 26338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:13.186394 26348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:13.186439 26352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:13.186357 26349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:13.187390 26338 server_base.cc:1034] running on GCE node
I20260810 06:55:13.188144 26338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:13.189471 26338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:13.190673 26338 hybrid_clock.cc:648] HybridClock initialized: now 1786344913190626 us; error 43 us; skew 500 ppm
I20260810 06:55:13.190989 26338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:13 dist-test-slave-f7th krb5kdc[26318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344913, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:13.198092 26338 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:13.199473 26338 webserver.cc:460] Webserver started at http://127.0.0.1:36485/ using document root <none> and password file <none>
I20260810 06:55:13.200080 26338 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:13.200198 26338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:13.200488 26338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:13.203348 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "0e1234db9f2a4a788b79cbd7166f5f3e"
format_stamp: "Formatted at 2026-08-10 06:55:13 on dist-test-slave-f7th"
server_key: "9dbd3309581d5ce3e51ef336bb6af572"
server_key_iv: "a6ad3926355a884d09e29667793fba6e"
server_key_version: "encryptionkey@0"
I20260810 06:55:13.204079 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "0e1234db9f2a4a788b79cbd7166f5f3e"
format_stamp: "Formatted at 2026-08-10 06:55:13 on dist-test-slave-f7th"
server_key: "9dbd3309581d5ce3e51ef336bb6af572"
server_key_iv: "a6ad3926355a884d09e29667793fba6e"
server_key_version: "encryptionkey@0"
I20260810 06:55:13.208966 26338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260810 06:55:13.212222 26361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:13.213441 26338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:55:13.213599 26338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "0e1234db9f2a4a788b79cbd7166f5f3e"
format_stamp: "Formatted at 2026-08-10 06:55:13 on dist-test-slave-f7th"
server_key: "9dbd3309581d5ce3e51ef336bb6af572"
server_key_iv: "a6ad3926355a884d09e29667793fba6e"
server_key_version: "encryptionkey@0"
I20260810 06:55:13.213768 26338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:13.228293 26338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:13.231793 26338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:13.232033 26338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:13.249940 26338 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46249
I20260810 06:55:13.249977 26439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46249 every 8 connection(s)
I20260810 06:55:13.251289 26338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:13.255752 26442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:13.261319 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26338
I20260810 06:55:13.261492 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:13.261958 22574 external_mini_cluster.cc:1428] Setting key b7971923723776c9cf34d91c9140df58
I20260810 06:55:13.268232 26442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e: Bootstrap starting.
I20260810 06:55:13.271725 26442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:13.273010 26442 log.cc:824] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:13.275465 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:13.263643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43091 (local address 127.0.0.1:46249)
0810 06:55:13.264391 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:13.264403 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:13.264442 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:55:13.266726 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:13.266750 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:13.267141 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:13.267453 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:13.268014 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:13.269419 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:13.271208 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:13.271885 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:13.272008 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:13.273205 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:13.273214 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:13.273227 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:13.273892 (+ 665us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:13.273979 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:13.274015 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:13.274149 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:13.274308 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 06:55:13.274576 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":313,"threads_started":1}
I20260810 06:55:13.275875 26442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e: No bootstrap required, opened a new log
I20260810 06:55:13.279350 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:13.284209 26442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260810 06:55:13.284547 26442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:13.284696 26442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1234db9f2a4a788b79cbd7166f5f3e, State: Initialized, Role: FOLLOWER
I20260810 06:55:13.285248 26442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260810 06:55:13.285516 26442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:13.285665 26442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:13.285847 26442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:13.288385 26442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260810 06:55:13.288923 26442 leader_election.cc:302] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e1234db9f2a4a788b79cbd7166f5f3e; no voters:
I20260810 06:55:13.289489 26442 leader_election.cc:288] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:13.289649 26461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:13.290446 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:13.283804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57563 (local address 127.0.0.1:46249)
0810 06:55:13.283985 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:13.283991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:13.284493 (+ 502us) server_negotiation.cc:407] Connection header received
0810 06:55:13.284704 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:13.284708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:13.284767 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:13.284855 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:13.285410 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:13.286426 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:13.287746 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:13.288401 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:13.288526 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:13.289616 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:13.289622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:13.289627 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:13.289650 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:13.289693 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:13.289714 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:13.289812 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:13.290030 (+ 218us) server_negotiation.cc:299] Negotiation successful
0810 06:55:13.290144 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:55:13.291133 26461 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [term 1 LEADER]: Becoming Leader. State: Replica: 0e1234db9f2a4a788b79cbd7166f5f3e, State: Running, Role: LEADER
I20260810 06:55:13.291698 26442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:13.291862 26461 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } }
I20260810 06:55:13.294996 26464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } } }
I20260810 06:55:13.295199 26468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e1234db9f2a4a788b79cbd7166f5f3e. Latest consensus state: current_term: 1 leader_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e1234db9f2a4a788b79cbd7166f5f3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46249 } } }
I20260810 06:55:13.295550 26468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:13.295550 26464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:13.297127 26479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:13.302282 26479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:13.310285 26479 catalog_manager.cc:1347] Generated new cluster ID: a829e88e81a44377b4798cf08c249a04
I20260810 06:55:13.310381 26479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:13.319612 26479 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:13.320802 26479 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:13.336323 26479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e1234db9f2a4a788b79cbd7166f5f3e: Generated new TSK 0
I20260810 06:55:13.337126 26479 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:13.347798 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26338
W20260810 06:55:13.348855 26431 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43091
W20260810 06:55:13.349465 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26338: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:13.349622 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26338
2026-08-10T06:55:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2052 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:13 dist-test-slave-f7th krb5kdc[26501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:13 dist-test-slave-f7th krb5kdc[26501](info): set up 2 sockets
Aug 10 06:55:13 dist-test-slave-f7th krb5kdc[26501](info): commencing operation
krb5kdc: starting...
W20260810 06:55:14.597178 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:14 dist-test-slave-f7th krb5kdc[26501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:14.883314 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42845
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:15.209502 26518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:15.209772 26518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:15.209878 26518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:15.216912 26518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:15.217007 26518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:15.217065 26518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:15.217123 26518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:15.217166 26518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:15.226441 26518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:15.227560 26518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:15.228796 26518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:15.239838 26529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:15.239748 26532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:15.239787 26530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:15.240895 26518 server_base.cc:1034] running on GCE node
I20260810 06:55:15.241602 26518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:15.242880 26518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:15.244076 26518 hybrid_clock.cc:648] HybridClock initialized: now 1786344915244042 us; error 34 us; skew 500 ppm
I20260810 06:55:15.244388 26518 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:15 dist-test-slave-f7th krb5kdc[26501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344915, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:15.248143 26518 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:15.249931 26518 webserver.cc:460] Webserver started at http://127.0.0.1:46841/ using document root <none> and password file <none>
I20260810 06:55:15.250694 26518 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:15.250813 26518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:15.251132 26518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:15.254307 26518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "74e3fc69abe54ff48ebd69aff67b5bd0"
format_stamp: "Formatted at 2026-08-10 06:55:15 on dist-test-slave-f7th"
server_key: "c08d69106772eb02e36ed0fe250dce29"
server_key_iv: "3c6576b9fbaa57b8b066c3a4ddde1385"
server_key_version: "encryptionkey@0"
I20260810 06:55:15.255048 26518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "74e3fc69abe54ff48ebd69aff67b5bd0"
format_stamp: "Formatted at 2026-08-10 06:55:15 on dist-test-slave-f7th"
server_key: "c08d69106772eb02e36ed0fe250dce29"
server_key_iv: "3c6576b9fbaa57b8b066c3a4ddde1385"
server_key_version: "encryptionkey@0"
I20260810 06:55:15.259810 26518 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 06:55:15.263084 26542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:15.264293 26518 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:55:15.264441 26518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "74e3fc69abe54ff48ebd69aff67b5bd0"
format_stamp: "Formatted at 2026-08-10 06:55:15 on dist-test-slave-f7th"
server_key: "c08d69106772eb02e36ed0fe250dce29"
server_key_iv: "3c6576b9fbaa57b8b066c3a4ddde1385"
server_key_version: "encryptionkey@0"
I20260810 06:55:15.264662 26518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:15.270051 26518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:15.273732 26518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:15.274000 26518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:15.290884 26518 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42845
I20260810 06:55:15.290946 26635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42845 every 8 connection(s)
I20260810 06:55:15.292152 26518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:15.295660 26637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:15.298264 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26518
I20260810 06:55:15.298424 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:15.298867 22574 external_mini_cluster.cc:1428] Setting key eaa7433a4d58c128c944fad40f27e403
I20260810 06:55:15.307279 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:15.300546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46175 (local address 127.0.0.1:42845)
0810 06:55:15.301375 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:15.301389 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:15.301438 (+ 49us) server_negotiation.cc:407] Connection header received
0810 06:55:15.303648 (+ 2210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:15.303676 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:15.304089 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:15.304494 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:15.305483 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:15.305496 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:15.305512 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:15.305529 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:15.305599 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:15.305626 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:15.305831 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:15.305969 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 06:55:15.306154 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":317,"threads_started":1}
I20260810 06:55:15.308296 26637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0: Bootstrap starting.
I20260810 06:55:15.310582 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:15.311436 26637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:15.312603 26637 log.cc:824] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:15.315686 26637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0: No bootstrap required, opened a new log
I20260810 06:55:15.317667 26642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:15.315493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45641 (local address 127.0.0.1:42845)
0810 06:55:15.315642 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:15.315647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:15.315947 (+ 300us) server_negotiation.cc:407] Connection header received
0810 06:55:15.316084 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:15.316089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:15.316147 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:15.316252 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:15.317077 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:15.317082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:15.317086 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:15.317103 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:15.317145 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:15.317169 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:15.317288 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:15.317334 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 06:55:15.317382 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:55:15.323770 26637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42845 } }
I20260810 06:55:15.324080 26637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:15.324151 26637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e3fc69abe54ff48ebd69aff67b5bd0, State: Initialized, Role: FOLLOWER
I20260810 06:55:15.324607 26637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42845 } }
I20260810 06:55:15.324842 26637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:15.324941 26637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:15.325057 26637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:15.327332 26637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42845 } }
I20260810 06:55:15.327754 26637 leader_election.cc:302] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74e3fc69abe54ff48ebd69aff67b5bd0; no voters:
I20260810 06:55:15.328238 26637 leader_election.cc:288] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:15.328351 26652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:15.329041 26652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [term 1 LEADER]: Becoming Leader. State: Replica: 74e3fc69abe54ff48ebd69aff67b5bd0, State: Running, Role: LEADER
I20260810 06:55:15.329519 26652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42845 } }
I20260810 06:55:15.330080 26637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:15.332463 26653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42845 } } }
I20260810 06:55:15.332909 26653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:15.332733 26655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e3fc69abe54ff48ebd69aff67b5bd0. Latest consensus state: current_term: 1 leader_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e3fc69abe54ff48ebd69aff67b5bd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42845 } } }
I20260810 06:55:15.333055 26655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:15.334622 26670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:15.339231 26670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:15.346772 26670 catalog_manager.cc:1347] Generated new cluster ID: 64944965cd2c41f4b684701d73e0d154
I20260810 06:55:15.346899 26670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:15.360456 26670 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:15.360850 26670 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:15.370307 26670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74e3fc69abe54ff48ebd69aff67b5bd0: Generated new TSK 0
I20260810 06:55:15.371066 26670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:15.386797 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26518
W20260810 06:55:15.387912 26627 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46175
W20260810 06:55:15.388500 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26518: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:15.388641 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26518
2026-08-10T06:55:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2036 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:15 dist-test-slave-f7th krb5kdc[26691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:15 dist-test-slave-f7th krb5kdc[26691](info): set up 2 sockets
Aug 10 06:55:15 dist-test-slave-f7th krb5kdc[26691](info): commencing operation
krb5kdc: starting...
W20260810 06:55:16.638979 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:16 dist-test-slave-f7th krb5kdc[26691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:16.925154 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:17.257522 26711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:17.257788 26711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:17.257946 26711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:17.264987 26711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:17.265097 26711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:17.265169 26711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:17.265251 26711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:17.265309 26711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:17.274878 26711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34529
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:17.276101 26711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:17.277428 26711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:17.288658 26725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:17.288781 26723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:17.288806 26721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:17.289630 26711 server_base.cc:1034] running on GCE node
I20260810 06:55:17.290386 26711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:17.291675 26711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:17.292869 26711 hybrid_clock.cc:648] HybridClock initialized: now 1786344917292824 us; error 32 us; skew 500 ppm
I20260810 06:55:17.293191 26711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:17 dist-test-slave-f7th krb5kdc[26691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:17.296130 26711 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:17.297521 26711 webserver.cc:460] Webserver started at http://127.0.0.1:43593/ using document root <none> and password file <none>
I20260810 06:55:17.298128 26711 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:17.298239 26711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:17.298519 26711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:17.301405 26711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "6d343c07ec7a46758526310e3e3bde2e"
format_stamp: "Formatted at 2026-08-10 06:55:17 on dist-test-slave-f7th"
server_key: "b9d1e19ded2d77a1b3469de25e53f372"
server_key_iv: "9bd5991ca89d186397d5e8bbbe24aac3"
server_key_version: "encryptionkey@0"
I20260810 06:55:17.302129 26711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "6d343c07ec7a46758526310e3e3bde2e"
format_stamp: "Formatted at 2026-08-10 06:55:17 on dist-test-slave-f7th"
server_key: "b9d1e19ded2d77a1b3469de25e53f372"
server_key_iv: "9bd5991ca89d186397d5e8bbbe24aac3"
server_key_version: "encryptionkey@0"
I20260810 06:55:17.306818 26711 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:55:17.310155 26738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:17.311355 26711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:55:17.311499 26711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "6d343c07ec7a46758526310e3e3bde2e"
format_stamp: "Formatted at 2026-08-10 06:55:17 on dist-test-slave-f7th"
server_key: "b9d1e19ded2d77a1b3469de25e53f372"
server_key_iv: "9bd5991ca89d186397d5e8bbbe24aac3"
server_key_version: "encryptionkey@0"
I20260810 06:55:17.311661 26711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:17.331658 26711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:17.335256 26711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:17.335496 26711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:17.352602 26711 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34529
I20260810 06:55:17.352651 26833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34529 every 8 connection(s)
I20260810 06:55:17.353885 26711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:17.357350 26834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:17.360790 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26711
I20260810 06:55:17.360972 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:17.361528 22574 external_mini_cluster.cc:1428] Setting key 93fbcbb7c7075d8b996cb7c87479d958
I20260810 06:55:17.369597 26834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e: Bootstrap starting.
I20260810 06:55:17.373229 26834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:17.374512 26834 log.cc:824] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:55:17 dist-test-slave-f7th krb5kdc[26691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344916, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:55:17.377537 26834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e: No bootstrap required, opened a new log
I20260810 06:55:17.385351 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:17.363363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:34045 (local address 127.0.0.1:34529)
0810 06:55:17.364068 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:17.364078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:17.364117 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:55:17.365738 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:17.365759 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:17.366145 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:17.366449 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:17.368183 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:17.369504 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:17.371200 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:17.371841 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:17.371961 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:17.376824 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:17.376835 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:17.376853 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:17.376904 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:17.380769 (+ 3865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:17.381564 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:17.381572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:17.381580 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:17.381795 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:17.382278 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:17.382283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:17.382287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:17.382962 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:17.383290 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:17.383652 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 06:55:17.384005 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":316,"threads_started":1}
I20260810 06:55:17.387300 26834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d343c07ec7a46758526310e3e3bde2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260810 06:55:17.387648 26834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:17.387761 26834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d343c07ec7a46758526310e3e3bde2e, State: Initialized, Role: FOLLOWER
I20260810 06:55:17.388285 26834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d343c07ec7a46758526310e3e3bde2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260810 06:55:17.388517 26834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:17.388671 26834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:17.388829 26834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:17.388937 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:17.391812 26834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d343c07ec7a46758526310e3e3bde2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260810 06:55:17.392381 26834 leader_election.cc:302] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d343c07ec7a46758526310e3e3bde2e; no voters:
I20260810 06:55:17.393095 26834 leader_election.cc:288] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:17.393205 26844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:17.395094 26844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [term 1 LEADER]: Becoming Leader. State: Replica: 6d343c07ec7a46758526310e3e3bde2e, State: Running, Role: LEADER
I20260810 06:55:17.395291 26834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:17.395743 26844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d343c07ec7a46758526310e3e3bde2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } }
I20260810 06:55:17.398605 26848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d343c07ec7a46758526310e3e3bde2e. Latest consensus state: current_term: 1 leader_uuid: "6d343c07ec7a46758526310e3e3bde2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d343c07ec7a46758526310e3e3bde2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } } }
I20260810 06:55:17.398974 26848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:17.399494 26847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d343c07ec7a46758526310e3e3bde2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d343c07ec7a46758526310e3e3bde2e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34529 } } }
I20260810 06:55:17.399892 26847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:17.413022 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:17.394483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:53141 (local address 127.0.0.1:34529)
0810 06:55:17.395154 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:17.395159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:17.395464 (+ 305us) server_negotiation.cc:407] Connection header received
0810 06:55:17.395519 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:17.395523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:17.395600 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:17.395692 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:17.397718 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:17.398903 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:17.400709 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:17.401393 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:17.401511 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:17.404886 (+ 3375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:17.404893 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:17.404899 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:17.404938 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:17.408902 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:17.409865 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:17.409871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:17.409875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:17.410022 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:17.410579 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:17.410585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:17.410589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:17.410987 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:17.411238 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:17.412413 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0810 06:55:17.412607 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548}
I20260810 06:55:17.414539 26862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:17.418809 26862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:17.426810 26862 catalog_manager.cc:1347] Generated new cluster ID: 0da8c374903d4dd9871a91593632b1b4
I20260810 06:55:17.426927 26862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:17.446664 26862 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:17.448452 26862 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:17.456223 26862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d343c07ec7a46758526310e3e3bde2e: Generated new TSK 0
I20260810 06:55:17.457196 26862 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:17.470063 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26711
I20260810 06:55:17.471487 26826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:17.561949 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26711
W20260810 06:55:17.587728 26686 connection.cc:535] client connection to 127.0.0.1:34529 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34529): Connection reset by peer (error 104)
2026-08-10T06:55:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2175 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:17 dist-test-slave-f7th krb5kdc[26882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:17 dist-test-slave-f7th krb5kdc[26882](info): set up 2 sockets
Aug 10 06:55:17 dist-test-slave-f7th krb5kdc[26882](info): commencing operation
krb5kdc: starting...
W20260810 06:55:18.821507 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:19 dist-test-slave-f7th krb5kdc[26882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:19.113097 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:19.442700 26905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:19.442975 26905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:19.443125 26905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:19.450155 26905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:19.450269 26905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:19.450346 26905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:19.450430 26905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:19.450492 26905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:19.460139 26905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:19.461455 26905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:19.462710 26905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:19.473868 26916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:19.473909 26918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:19.473949 26922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:19.474629 26905 server_base.cc:1034] running on GCE node
I20260810 06:55:19.475338 26905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:19.476595 26905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:19.477823 26905 hybrid_clock.cc:648] HybridClock initialized: now 1786344919477782 us; error 32 us; skew 500 ppm
I20260810 06:55:19.478152 26905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:19 dist-test-slave-f7th krb5kdc[26882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:19.485010 26905 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:19.486326 26905 webserver.cc:460] Webserver started at http://127.0.0.1:33691/ using document root <none> and password file <none>
I20260810 06:55:19.486963 26905 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:19.487107 26905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:19.487385 26905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:19.490291 26905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "86cc11386235475e8d44d4a70109f24a"
format_stamp: "Formatted at 2026-08-10 06:55:19 on dist-test-slave-f7th"
server_key: "facb1903fa748437f22b6d8f4b9316c9"
server_key_iv: "ca944b27b59b5be70bdcdeb84c320198"
server_key_version: "encryptionkey@0"
I20260810 06:55:19.491017 26905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "86cc11386235475e8d44d4a70109f24a"
format_stamp: "Formatted at 2026-08-10 06:55:19 on dist-test-slave-f7th"
server_key: "facb1903fa748437f22b6d8f4b9316c9"
server_key_iv: "ca944b27b59b5be70bdcdeb84c320198"
server_key_version: "encryptionkey@0"
I20260810 06:55:19.495728 26905 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:55:19.498898 26938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:19.500123 26905 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:55:19.500281 26905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "86cc11386235475e8d44d4a70109f24a"
format_stamp: "Formatted at 2026-08-10 06:55:19 on dist-test-slave-f7th"
server_key: "facb1903fa748437f22b6d8f4b9316c9"
server_key_iv: "ca944b27b59b5be70bdcdeb84c320198"
server_key_version: "encryptionkey@0"
I20260810 06:55:19.500463 26905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:19.543852 26905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:19.547963 26905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:19.548233 26905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:19.565088 26905 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41441
I20260810 06:55:19.565119 27027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41441 every 8 connection(s)
I20260810 06:55:19.566327 26905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:19.569952 27030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:19.572827 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 26905
I20260810 06:55:19.573024 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:19.573591 22574 external_mini_cluster.cc:1428] Setting key d0e13329d05eae1dd80147a561b93ce3
I20260810 06:55:19.581239 27030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a: Bootstrap starting.
I20260810 06:55:19.584198 27030 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:19.585314 27030 log.cc:824] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:19.586304 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:19.575496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:60307 (local address 127.0.0.1:41441)
0810 06:55:19.576245 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:19.576257 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:19.576304 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:55:19.577972 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:19.577997 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:19.578384 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:19.578731 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:19.579281 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:19.580546 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:19.581945 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:19.582611 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:19.582780 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:19.583957 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:19.583965 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:19.583977 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:19.584508 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:19.584589 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:19.584636 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:19.584771 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:19.585039 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 06:55:19.585196 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":315,"threads_started":1}
I20260810 06:55:19.588217 27030 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a: No bootstrap required, opened a new log
I20260810 06:55:19.590117 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:19.596016 27030 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc11386235475e8d44d4a70109f24a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41441 } }
I20260810 06:55:19.596451 27030 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:19.596561 27030 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86cc11386235475e8d44d4a70109f24a, State: Initialized, Role: FOLLOWER
I20260810 06:55:19.597195 27030 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc11386235475e8d44d4a70109f24a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41441 } }
I20260810 06:55:19.597477 27030 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:19.597615 27030 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:19.597774 27030 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:19.600524 27030 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc11386235475e8d44d4a70109f24a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41441 } }
I20260810 06:55:19.600917 27036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:19.594321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:50793 (local address 127.0.0.1:41441)
0810 06:55:19.594504 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:19.594509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:19.594822 (+ 313us) server_negotiation.cc:407] Connection header received
0810 06:55:19.594980 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:19.594984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:19.595043 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:19.595125 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:19.595688 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:19.596810 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:19.598566 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:19.599048 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:19.599144 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:19.600175 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:19.600180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:19.600183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:19.600198 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:19.600249 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:19.600263 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:19.600338 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:19.600572 (+ 234us) server_negotiation.cc:299] Negotiation successful
0810 06:55:19.600633 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 06:55:19.601126 27030 leader_election.cc:302] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86cc11386235475e8d44d4a70109f24a; no voters:
I20260810 06:55:19.601747 27030 leader_election.cc:288] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:19.601914 27047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:19.603742 27030 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:19.608529 27047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [term 1 LEADER]: Becoming Leader. State: Replica: 86cc11386235475e8d44d4a70109f24a, State: Running, Role: LEADER
I20260810 06:55:19.608994 27047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc11386235475e8d44d4a70109f24a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41441 } }
I20260810 06:55:19.611594 27048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86cc11386235475e8d44d4a70109f24a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc11386235475e8d44d4a70109f24a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41441 } } }
I20260810 06:55:19.611730 27051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86cc11386235475e8d44d4a70109f24a. Latest consensus state: current_term: 1 leader_uuid: "86cc11386235475e8d44d4a70109f24a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86cc11386235475e8d44d4a70109f24a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41441 } } }
I20260810 06:55:19.612015 27048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:19.612030 27051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a [sys.catalog]: This master's current role is: LEADER
W20260810 06:55:19.617182 27072 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:55:19.617290 27072 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:55:19.617429 27073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:19.619514 27073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:19.627677 27073 catalog_manager.cc:1347] Generated new cluster ID: 1c84ac6ac87742898b72805315d496e5
I20260810 06:55:19.627772 27073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:19.648011 27073 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:19.649326 27073 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:19.655536 27073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86cc11386235475e8d44d4a70109f24a: Generated new TSK 0
I20260810 06:55:19.656493 27073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:19.662388 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26905
W20260810 06:55:19.663388 27016 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:60307
W20260810 06:55:19.664116 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 26905: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:19.664247 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 26905
W20260810 06:55:19.692019 26880 connection.cc:535] client connection to 127.0.0.1:41441 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41441): Connection reset by peer (error 104)
2026-08-10T06:55:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2104 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:19 dist-test-slave-f7th krb5kdc[27082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:19 dist-test-slave-f7th krb5kdc[27082](info): set up 2 sockets
Aug 10 06:55:19 dist-test-slave-f7th krb5kdc[27082](info): commencing operation
krb5kdc: starting...
W20260810 06:55:20.920588 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.006s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:21 dist-test-slave-f7th krb5kdc[27082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:21.203768 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:21.532886 27101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:21.533174 27101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:21.533313 27101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:21.541453 27101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:21.541579 27101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:21.541661 27101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:21.541744 27101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:21.541803 27101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:21.551561 27101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:21.552825 27101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:21.554083 27101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:21.569140 27107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:21.569257 27109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:21.569630 27113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:21.570598 27101 server_base.cc:1034] running on GCE node
I20260810 06:55:21.571344 27101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:21.572741 27101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:21.573949 27101 hybrid_clock.cc:648] HybridClock initialized: now 1786344921573897 us; error 49 us; skew 500 ppm
I20260810 06:55:21.574272 27101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:21 dist-test-slave-f7th krb5kdc[27082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344921, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:21.577277 27101 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:21.578619 27101 webserver.cc:460] Webserver started at http://127.0.0.1:37669/ using document root <none> and password file <none>
I20260810 06:55:21.579278 27101 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:21.579393 27101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:21.579691 27101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:21.582583 27101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "e09fa414082e44f6a5c9fb5b40869202"
format_stamp: "Formatted at 2026-08-10 06:55:21 on dist-test-slave-f7th"
server_key: "07ab11f466f39af40d0e20b7d9663e58"
server_key_iv: "3677ed71c344c32187d22dd109be0331"
server_key_version: "encryptionkey@0"
I20260810 06:55:21.583298 27101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "e09fa414082e44f6a5c9fb5b40869202"
format_stamp: "Formatted at 2026-08-10 06:55:21 on dist-test-slave-f7th"
server_key: "07ab11f466f39af40d0e20b7d9663e58"
server_key_iv: "3677ed71c344c32187d22dd109be0331"
server_key_version: "encryptionkey@0"
I20260810 06:55:21.588052 27101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260810 06:55:21.591267 27136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:21.592419 27101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 06:55:21.592599 27101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "e09fa414082e44f6a5c9fb5b40869202"
format_stamp: "Formatted at 2026-08-10 06:55:21 on dist-test-slave-f7th"
server_key: "07ab11f466f39af40d0e20b7d9663e58"
server_key_iv: "3677ed71c344c32187d22dd109be0331"
server_key_version: "encryptionkey@0"
I20260810 06:55:21.592824 27101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:21.598148 27101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:21.601684 27101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:21.601940 27101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:21.618801 27101 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42103
I20260810 06:55:21.618840 27224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42103 every 8 connection(s)
I20260810 06:55:21.620059 27101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:21.623535 27225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:21.629873 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27101
I20260810 06:55:21.630043 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:21.630460 22574 external_mini_cluster.cc:1428] Setting key 2d813bde4cd9b0de27240a9df34c1472
I20260810 06:55:21.633533 27225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202: Bootstrap starting.
I20260810 06:55:21.637516 27225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:21.638865 27225 log.cc:824] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:21.639741 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:21.632344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:36441 (local address 127.0.0.1:42103)
0810 06:55:21.633266 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:21.633282 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:21.633615 (+ 333us) server_negotiation.cc:407] Connection header received
0810 06:55:21.636055 (+ 2440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:21.636083 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:21.636479 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:21.636951 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:21.637923 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:21.637931 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:21.637953 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:21.637972 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:21.638049 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:21.638075 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:21.638247 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:21.638382 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:55:21.638582 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":405,"threads_started":1}
I20260810 06:55:21.641845 27225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202: No bootstrap required, opened a new log
I20260810 06:55:21.643339 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:21.650868 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:21.648523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:46635 (local address 127.0.0.1:42103)
0810 06:55:21.648716 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:21.648722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:21.649094 (+ 372us) server_negotiation.cc:407] Connection header received
0810 06:55:21.649227 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:21.649233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:21.649292 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:21.649395 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:21.650244 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:21.650250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:21.650254 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:21.650273 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:21.650324 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:21.650357 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:21.650478 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:21.650520 (+ 42us) server_negotiation.cc:299] Negotiation successful
0810 06:55:21.650584 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:55:21.651685 27225 raft_consensus.cc:348] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09fa414082e44f6a5c9fb5b40869202" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42103 } }
I20260810 06:55:21.652139 27225 raft_consensus.cc:374] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:21.652256 27225 raft_consensus.cc:729] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e09fa414082e44f6a5c9fb5b40869202, State: Initialized, Role: FOLLOWER
I20260810 06:55:21.652810 27225 consensus_queue.cc:260] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09fa414082e44f6a5c9fb5b40869202" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42103 } }
I20260810 06:55:21.653026 27225 raft_consensus.cc:388] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:21.653136 27225 raft_consensus.cc:482] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:21.653265 27225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:21.655642 27225 raft_consensus.cc:504] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09fa414082e44f6a5c9fb5b40869202" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42103 } }
I20260810 06:55:21.656083 27225 leader_election.cc:302] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e09fa414082e44f6a5c9fb5b40869202; no voters:
I20260810 06:55:21.656582 27225 leader_election.cc:288] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:21.656726 27242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:21.657437 27242 raft_consensus.cc:686] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [term 1 LEADER]: Becoming Leader. State: Replica: e09fa414082e44f6a5c9fb5b40869202, State: Running, Role: LEADER
I20260810 06:55:21.658028 27242 consensus_queue.cc:237] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09fa414082e44f6a5c9fb5b40869202" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42103 } }
I20260810 06:55:21.658710 27225 sys_catalog.cc:564] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:21.660873 27243 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e09fa414082e44f6a5c9fb5b40869202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09fa414082e44f6a5c9fb5b40869202" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42103 } } }
I20260810 06:55:21.661332 27243 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:21.661183 27244 sys_catalog.cc:455] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e09fa414082e44f6a5c9fb5b40869202. Latest consensus state: current_term: 1 leader_uuid: "e09fa414082e44f6a5c9fb5b40869202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e09fa414082e44f6a5c9fb5b40869202" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42103 } } }
I20260810 06:55:21.661466 27244 sys_catalog.cc:458] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:21.663061 27254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:21.667559 27254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:21.674962 27254 catalog_manager.cc:1347] Generated new cluster ID: 1d2047222f2b4cc883a680e8ee502c4d
I20260810 06:55:21.675069 27254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:21.683341 27254 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:21.683755 27254 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:21.697615 27254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e09fa414082e44f6a5c9fb5b40869202: Generated new TSK 0
I20260810 06:55:21.698364 27254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:21.708253 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27101
W20260810 06:55:21.709280 27218 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:36441
W20260810 06:55:21.709898 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27101: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:21.710023 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27101
2026-08-10T06:55:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2047 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:21 dist-test-slave-f7th krb5kdc[27275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:21 dist-test-slave-f7th krb5kdc[27275](info): set up 2 sockets
Aug 10 06:55:21 dist-test-slave-f7th krb5kdc[27275](info): commencing operation
krb5kdc: starting...
W20260810 06:55:22.971144 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:23 dist-test-slave-f7th krb5kdc[27275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:23.262852 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40543
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:23.589332 27293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:23.589685 27293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:23.589833 27293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:23.599030 27293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:23.599140 27293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:23.599215 27293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:23.599282 27293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:23.599326 27293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:23.609030 27293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40543
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:23.610234 27293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:23.611505 27293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:23.622964 27303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:23.623009 27306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:23.622922 27310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:23.623808 27293 server_base.cc:1034] running on GCE node
I20260810 06:55:23.624495 27293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:23.625974 27293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:23.627163 27293 hybrid_clock.cc:648] HybridClock initialized: now 1786344923627112 us; error 47 us; skew 500 ppm
I20260810 06:55:23.627491 27293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:23 dist-test-slave-f7th krb5kdc[27275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344923, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:23.630426 27293 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:23.631786 27293 webserver.cc:460] Webserver started at http://127.0.0.1:36409/ using document root <none> and password file <none>
I20260810 06:55:23.632387 27293 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:23.632486 27293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:23.632779 27293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:23.635596 27293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7"
format_stamp: "Formatted at 2026-08-10 06:55:23 on dist-test-slave-f7th"
server_key: "bf2a4935527d3b6db4707d3258d18928"
server_key_iv: "9863b53f0625965b15c5183e5128daba"
server_key_version: "encryptionkey@0"
I20260810 06:55:23.636302 27293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7"
format_stamp: "Formatted at 2026-08-10 06:55:23 on dist-test-slave-f7th"
server_key: "bf2a4935527d3b6db4707d3258d18928"
server_key_iv: "9863b53f0625965b15c5183e5128daba"
server_key_version: "encryptionkey@0"
I20260810 06:55:23.641291 27293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 06:55:23.644595 27330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:23.645857 27293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 06:55:23.645999 27293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7"
format_stamp: "Formatted at 2026-08-10 06:55:23 on dist-test-slave-f7th"
server_key: "bf2a4935527d3b6db4707d3258d18928"
server_key_iv: "9863b53f0625965b15c5183e5128daba"
server_key_version: "encryptionkey@0"
I20260810 06:55:23.646154 27293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:23.665308 27293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:23.668974 27293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:23.669211 27293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:23.686518 27293 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40543
I20260810 06:55:23.686555 27422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40543 every 8 connection(s)
I20260810 06:55:23.687819 27293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:23.688504 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27293
I20260810 06:55:23.688689 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:23.689229 22574 external_mini_cluster.cc:1428] Setting key 9500631f785711479e5a571872fba302
I20260810 06:55:23.691951 27425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:55:23 dist-test-slave-f7th krb5kdc[27275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 06:55:23.702081 27425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7: Bootstrap starting.
I20260810 06:55:23.705511 27425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:23.706588 27425 log.cc:824] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:23.709666 27425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7: No bootstrap required, opened a new log
I20260810 06:55:23.710407 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:23.690923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:51107 (local address 127.0.0.1:40543)
0810 06:55:23.691639 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:23.691650 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:23.691689 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:55:23.693323 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:23.693345 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:23.693700 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:23.694040 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:23.695466 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:23.696851 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:23.698066 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:23.698705 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:23.698866 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:23.703310 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:23.703317 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:23.703330 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:23.703365 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:23.706317 (+ 2952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:23.707280 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:23.707289 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:23.707296 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:23.707518 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:23.707932 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:23.707936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:23.707938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:23.708400 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:23.708657 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:23.709149 (+ 492us) server_negotiation.cc:299] Negotiation successful
0810 06:55:23.709376 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":305,"threads_started":1}
I20260810 06:55:23.714152 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:23.717557 27425 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40543 } }
I20260810 06:55:23.717875 27425 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:23.717959 27425 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3891fe3a0984e46a0b0af66ff3f2bd7, State: Initialized, Role: FOLLOWER
I20260810 06:55:23.718580 27425 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40543 } }
I20260810 06:55:23.718797 27425 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:23.718923 27425 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:23.719118 27425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:23.721698 27425 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40543 } }
I20260810 06:55:23.722220 27425 leader_election.cc:302] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3891fe3a0984e46a0b0af66ff3f2bd7; no voters:
I20260810 06:55:23.722755 27425 leader_election.cc:288] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:23.722883 27443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:23.724663 27425 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:23.725415 27443 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [term 1 LEADER]: Becoming Leader. State: Replica: d3891fe3a0984e46a0b0af66ff3f2bd7, State: Running, Role: LEADER
I20260810 06:55:23.726063 27443 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40543 } }
I20260810 06:55:23.728731 27446 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3891fe3a0984e46a0b0af66ff3f2bd7. Latest consensus state: current_term: 1 leader_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40543 } } }
I20260810 06:55:23.729144 27446 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:23.729494 27445 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3891fe3a0984e46a0b0af66ff3f2bd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40543 } } }
I20260810 06:55:23.729851 27445 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:23.730183 27456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:23.737975 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:23.718283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:57605 (local address 127.0.0.1:40543)
0810 06:55:23.718474 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:23.718481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:23.718966 (+ 485us) server_negotiation.cc:407] Connection header received
0810 06:55:23.719114 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:23.719119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:23.719197 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:23.719300 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:23.720594 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:23.721726 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:23.723398 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:23.724160 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:23.724265 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:23.727659 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:23.727664 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:23.727668 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:23.727700 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:23.733042 (+ 5342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:23.734169 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:23.734175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:23.734179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:23.734518 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:23.735264 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:23.735270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:23.735274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:23.735676 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:23.737207 (+ 1531us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:23.737384 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 06:55:23.737579 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":54}
I20260810 06:55:23.747263 27456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:23.756029 27456 catalog_manager.cc:1347] Generated new cluster ID: 2e54a32554ac4dc09a17541542cd20cf
I20260810 06:55:23.756167 27456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:23.770234 27456 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:23.771955 27456 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:23.780203 27456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3891fe3a0984e46a0b0af66ff3f2bd7: Generated new TSK 0
I20260810 06:55:23.781195 27456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:23.792735 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27293
I20260810 06:55:23.794464 27416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:23.883360 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27293
W20260810 06:55:23.909286 27270 connection.cc:535] client connection to 127.0.0.1:40543 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40543): Connection reset by peer (error 104)
2026-08-10T06:55:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2169 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:23 dist-test-slave-f7th krb5kdc[27483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:23 dist-test-slave-f7th krb5kdc[27483](info): set up 2 sockets
Aug 10 06:55:23 dist-test-slave-f7th krb5kdc[27483](info): commencing operation
krb5kdc: starting...
W20260810 06:55:25.143659 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:25 dist-test-slave-f7th krb5kdc[27483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:25.431130 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:25.760090 27504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:25.760358 27504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:25.760492 27504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:25.767570 27504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:25.767670 27504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:25.767748 27504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:25.767840 27504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:25.767908 27504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:25.777334 27504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:25.778493 27504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:25.779717 27504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:25.790776 27521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:25.790738 27512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:25.790808 27514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:25.791538 27504 server_base.cc:1034] running on GCE node
I20260810 06:55:25.792270 27504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:25.793589 27504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:25.794833 27504 hybrid_clock.cc:648] HybridClock initialized: now 1786344925794772 us; error 36 us; skew 500 ppm
I20260810 06:55:25.795207 27504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:25 dist-test-slave-f7th krb5kdc[27483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:25.801403 27504 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:25.802781 27504 webserver.cc:460] Webserver started at http://127.0.0.1:33029/ using document root <none> and password file <none>
I20260810 06:55:25.803382 27504 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:25.803481 27504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:25.803720 27504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:25.806589 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "9171bcc81d8e4ce1a033052ef119ef27"
format_stamp: "Formatted at 2026-08-10 06:55:25 on dist-test-slave-f7th"
server_key: "c5c53ae391cd2a97201ee115f259b388"
server_key_iv: "4c6d44d194960842b31a3178c7f5e62f"
server_key_version: "encryptionkey@0"
I20260810 06:55:25.807292 27504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "9171bcc81d8e4ce1a033052ef119ef27"
format_stamp: "Formatted at 2026-08-10 06:55:25 on dist-test-slave-f7th"
server_key: "c5c53ae391cd2a97201ee115f259b388"
server_key_iv: "4c6d44d194960842b31a3178c7f5e62f"
server_key_version: "encryptionkey@0"
I20260810 06:55:25.812005 27504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260810 06:55:25.815142 27536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:25.816334 27504 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260810 06:55:25.816488 27504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "9171bcc81d8e4ce1a033052ef119ef27"
format_stamp: "Formatted at 2026-08-10 06:55:25 on dist-test-slave-f7th"
server_key: "c5c53ae391cd2a97201ee115f259b388"
server_key_iv: "4c6d44d194960842b31a3178c7f5e62f"
server_key_version: "encryptionkey@0"
I20260810 06:55:25.816685 27504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:25.828570 27504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:25.832139 27504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:25.832381 27504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:25.849400 27504 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38143
I20260810 06:55:25.849407 27632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38143 every 8 connection(s)
I20260810 06:55:25.850678 27504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:25.854208 27635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:25.856153 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27504
I20260810 06:55:25.856324 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:25.856801 22574 external_mini_cluster.cc:1428] Setting key efef10c9bbe700bd0a34cb3fd87399a2
I20260810 06:55:25.865890 27635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27: Bootstrap starting.
I20260810 06:55:25.869364 27635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:25.870102 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:25.858596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:55951 (local address 127.0.0.1:38143)
0810 06:55:25.859414 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:25.859426 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:25.859473 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:55:25.861198 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:25.861220 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:25.861610 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:25.861919 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:25.862507 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:25.863835 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:25.865680 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:25.866385 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:25.866515 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:25.867807 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:25.867814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:25.867827 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:25.868349 (+ 522us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:25.868426 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:25.868451 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:25.868565 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:25.868878 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 06:55:25.869056 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":367,"threads_started":1}
I20260810 06:55:25.870461 27635 log.cc:824] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:25.873432 27635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27: No bootstrap required, opened a new log
I20260810 06:55:25.873962 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:25.882962 27635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9171bcc81d8e4ce1a033052ef119ef27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38143 } }
I20260810 06:55:25.883383 27635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:25.883492 27635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9171bcc81d8e4ce1a033052ef119ef27, State: Initialized, Role: FOLLOWER
I20260810 06:55:25.884086 27635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9171bcc81d8e4ce1a033052ef119ef27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38143 } }
I20260810 06:55:25.884302 27635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:25.884426 27635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:25.884645 27635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:25.885357 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:25.879179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:45089 (local address 127.0.0.1:38143)
0810 06:55:25.879344 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:25.879349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:25.879651 (+ 302us) server_negotiation.cc:407] Connection header received
0810 06:55:25.879803 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:25.879808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:25.879880 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:25.879993 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:25.880425 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:25.881571 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:25.882793 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:25.883424 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:25.883554 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:25.884555 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:25.884561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:25.884566 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:25.884591 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:25.884659 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:25.884684 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:25.884806 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:25.884979 (+ 173us) server_negotiation.cc:299] Negotiation successful
0810 06:55:25.885047 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:55:25.887313 27635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9171bcc81d8e4ce1a033052ef119ef27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38143 } }
I20260810 06:55:25.887737 27635 leader_election.cc:302] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9171bcc81d8e4ce1a033052ef119ef27; no voters:
I20260810 06:55:25.888226 27635 leader_election.cc:288] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:25.888346 27655 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:25.889048 27655 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [term 1 LEADER]: Becoming Leader. State: Replica: 9171bcc81d8e4ce1a033052ef119ef27, State: Running, Role: LEADER
I20260810 06:55:25.889573 27655 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9171bcc81d8e4ce1a033052ef119ef27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38143 } }
I20260810 06:55:25.890141 27635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:25.892211 27656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9171bcc81d8e4ce1a033052ef119ef27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9171bcc81d8e4ce1a033052ef119ef27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38143 } } }
I20260810 06:55:25.892562 27656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:25.893328 27657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9171bcc81d8e4ce1a033052ef119ef27. Latest consensus state: current_term: 1 leader_uuid: "9171bcc81d8e4ce1a033052ef119ef27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9171bcc81d8e4ce1a033052ef119ef27" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38143 } } }
I20260810 06:55:25.893654 27657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:25.897917 27662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:25.904054 27662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:25.912740 27662 catalog_manager.cc:1347] Generated new cluster ID: e05e208072824782b649b6b2e65e8785
I20260810 06:55:25.912834 27662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:25.921361 27662 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:25.922631 27662 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:25.931560 27662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9171bcc81d8e4ce1a033052ef119ef27: Generated new TSK 0
I20260810 06:55:25.932272 27662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:25.944551 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27504
W20260810 06:55:25.946317 27621 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:55951
W20260810 06:55:25.946971 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27504: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:25.947105 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27504
W20260810 06:55:25.971369 27476 connection.cc:535] client connection to 127.0.0.1:38143 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38143): Connection reset by peer (error 104)
2026-08-10T06:55:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2062 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:26 dist-test-slave-f7th krb5kdc[27685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:26 dist-test-slave-f7th krb5kdc[27685](info): set up 2 sockets
Aug 10 06:55:26 dist-test-slave-f7th krb5kdc[27685](info): commencing operation
krb5kdc: starting...
W20260810 06:55:27.194062 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:27 dist-test-slave-f7th krb5kdc[27685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 06:55:27.476341 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35871
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:27.803378 27718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:27.803633 27718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:27.803761 27718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:27.810760 27718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:27.810853 27718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:27.810928 27718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:27.811014 27718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:27.811069 27718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:27.820381 27718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35871
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:27.821539 27718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:27.822767 27718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:27.833905 27731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:27.834015 27733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:27.834051 27730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:27.834860 27718 server_base.cc:1034] running on GCE node
I20260810 06:55:27.835603 27718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:27.836910 27718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:27.838096 27718 hybrid_clock.cc:648] HybridClock initialized: now 1786344927838048 us; error 47 us; skew 500 ppm
I20260810 06:55:27.838403 27718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:27 dist-test-slave-f7th krb5kdc[27685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344927, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:27.841321 27718 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 06:55:27.842718 27718 webserver.cc:460] Webserver started at http://127.0.0.1:35717/ using document root <none> and password file <none>
I20260810 06:55:27.843350 27718 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:27.843466 27718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:27.843752 27718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:27.847035 27718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "230bbe6754a2422da7102b5a712dce3a"
format_stamp: "Formatted at 2026-08-10 06:55:27 on dist-test-slave-f7th"
server_key: "4a70fcc571d17a8803fe09fcb98b14b1"
server_key_iv: "82ed939c8f60d1861e78618fa1f7bf75"
server_key_version: "encryptionkey@0"
I20260810 06:55:27.847757 27718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "230bbe6754a2422da7102b5a712dce3a"
format_stamp: "Formatted at 2026-08-10 06:55:27 on dist-test-slave-f7th"
server_key: "4a70fcc571d17a8803fe09fcb98b14b1"
server_key_iv: "82ed939c8f60d1861e78618fa1f7bf75"
server_key_version: "encryptionkey@0"
I20260810 06:55:27.852517 27718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260810 06:55:27.855690 27742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:27.856885 27718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260810 06:55:27.857046 27718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "230bbe6754a2422da7102b5a712dce3a"
format_stamp: "Formatted at 2026-08-10 06:55:27 on dist-test-slave-f7th"
server_key: "4a70fcc571d17a8803fe09fcb98b14b1"
server_key_iv: "82ed939c8f60d1861e78618fa1f7bf75"
server_key_version: "encryptionkey@0"
I20260810 06:55:27.857210 27718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:27.862571 27718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:27.866142 27718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:27.866393 27718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:27.883424 27718 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35871
I20260810 06:55:27.883481 27825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35871 every 8 connection(s)
I20260810 06:55:27.884788 27718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:27.888288 27826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:27.890833 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27718
I20260810 06:55:27.891005 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:27.891450 22574 external_mini_cluster.cc:1428] Setting key 605ad6ef5bfb50a229d423d693a13e9b
I20260810 06:55:27.899976 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:27.893298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:48719 (local address 127.0.0.1:35871)
0810 06:55:27.894087 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:27.894101 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:27.894147 (+ 46us) server_negotiation.cc:407] Connection header received
0810 06:55:27.896354 (+ 2207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:27.896383 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:27.896841 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:27.897253 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:27.898189 (+ 936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:27.898196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:27.898211 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:27.898236 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:27.898301 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:27.898325 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:27.898553 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:27.898682 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 06:55:27.898862 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":322,"threads_started":1}
I20260810 06:55:27.900825 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a: Bootstrap starting.
I20260810 06:55:27.903281 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:27.903863 27826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:27.904858 27826 log.cc:824] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:27.907564 27826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a: No bootstrap required, opened a new log
I20260810 06:55:27.909906 27831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:27.907473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.209:53587 (local address 127.0.0.1:35871)
0810 06:55:27.907649 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:27.907656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:27.908118 (+ 462us) server_negotiation.cc:407] Connection header received
0810 06:55:27.908280 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:27.908283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:27.908326 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:27.908399 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:27.909320 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:27.909324 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:27.909327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:27.909341 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:27.909377 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:27.909393 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:27.909481 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:27.909595 (+ 114us) server_negotiation.cc:299] Negotiation successful
0810 06:55:27.909643 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 06:55:27.915297 27826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230bbe6754a2422da7102b5a712dce3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260810 06:55:27.915611 27826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:27.915700 27826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 230bbe6754a2422da7102b5a712dce3a, State: Initialized, Role: FOLLOWER
I20260810 06:55:27.916200 27826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230bbe6754a2422da7102b5a712dce3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260810 06:55:27.916424 27826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:27.916553 27826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:27.916775 27826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:27.918854 27826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230bbe6754a2422da7102b5a712dce3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260810 06:55:27.919296 27826 leader_election.cc:302] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 230bbe6754a2422da7102b5a712dce3a; no voters:
I20260810 06:55:27.919799 27826 leader_election.cc:288] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:27.919905 27843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:27.920478 27843 raft_consensus.cc:686] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [term 1 LEADER]: Becoming Leader. State: Replica: 230bbe6754a2422da7102b5a712dce3a, State: Running, Role: LEADER
I20260810 06:55:27.920989 27843 consensus_queue.cc:237] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230bbe6754a2422da7102b5a712dce3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } }
I20260810 06:55:27.921617 27826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:27.923525 27846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "230bbe6754a2422da7102b5a712dce3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230bbe6754a2422da7102b5a712dce3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } } }
I20260810 06:55:27.923935 27846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:27.926561 27850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 230bbe6754a2422da7102b5a712dce3a. Latest consensus state: current_term: 1 leader_uuid: "230bbe6754a2422da7102b5a712dce3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "230bbe6754a2422da7102b5a712dce3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35871 } } }
I20260810 06:55:27.926791 27850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:27.927438 27858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:27.931814 27858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:27.939559 27858 catalog_manager.cc:1347] Generated new cluster ID: 2b827defdb7d48af88679e28d1f0f1e0
I20260810 06:55:27.939663 27858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:27.955147 27858 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:27.955601 27858 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:27.965716 27858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 230bbe6754a2422da7102b5a712dce3a: Generated new TSK 0
I20260810 06:55:27.966612 27858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:27.974516 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27718
W20260810 06:55:27.975476 27819 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.209:48719
W20260810 06:55:27.976092 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27718: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:27.976218 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27718
2026-08-10T06:55:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2031 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:28 dist-test-slave-f7th krb5kdc[27884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:28 dist-test-slave-f7th krb5kdc[27884](info): set up 2 sockets
Aug 10 06:55:28 dist-test-slave-f7th krb5kdc[27884](info): commencing operation
krb5kdc: starting...
W20260810 06:55:29.238780 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:29 dist-test-slave-f7th krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:29.541934 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34347
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:34347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:29.888358 27906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:29.888687 27906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:29.888831 27906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:29.896287 27906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:29.896406 27906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:29.896490 27906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:29.896574 27906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:29.896661 27906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:29.906720 27906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:34347
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:34347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:29.908015 27906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:29.909394 27906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:29.920692 27922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:29.920768 27928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:29.920789 27924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:29.921947 27906 server_base.cc:1034] running on GCE node
I20260810 06:55:29.922735 27906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:29.924038 27906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:29.925249 27906 hybrid_clock.cc:648] HybridClock initialized: now 1786344929925195 us; error 43 us; skew 500 ppm
I20260810 06:55:29.925602 27906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:29 dist-test-slave-f7th krb5kdc[27884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344929, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:29.928661 27906 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:29.930135 27906 webserver.cc:460] Webserver started at http://127.22.11.190:33171/ using document root <none> and password file <none>
I20260810 06:55:29.930886 27906 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:29.931022 27906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:29.931314 27906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:29.934207 27906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "d9daf3acf0394403be843667aefba795"
format_stamp: "Formatted at 2026-08-10 06:55:29 on dist-test-slave-f7th"
server_key: "00258dcab9282d5327806c1b3cfb61a9"
server_key_iv: "8bd75b223658ef89dcbb7d29a38b24bf"
server_key_version: "encryptionkey@0"
I20260810 06:55:29.934914 27906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "d9daf3acf0394403be843667aefba795"
format_stamp: "Formatted at 2026-08-10 06:55:29 on dist-test-slave-f7th"
server_key: "00258dcab9282d5327806c1b3cfb61a9"
server_key_iv: "8bd75b223658ef89dcbb7d29a38b24bf"
server_key_version: "encryptionkey@0"
I20260810 06:55:29.939783 27906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 06:55:29.943220 27937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:29.944502 27906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 06:55:29.944736 27906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "d9daf3acf0394403be843667aefba795"
format_stamp: "Formatted at 2026-08-10 06:55:29 on dist-test-slave-f7th"
server_key: "00258dcab9282d5327806c1b3cfb61a9"
server_key_iv: "8bd75b223658ef89dcbb7d29a38b24bf"
server_key_version: "encryptionkey@0"
I20260810 06:55:29.944918 27906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:29.970860 27906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:29.975111 27906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:29.975373 27906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:29.992697 27906 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:34347
I20260810 06:55:29.992740 28028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:34347 every 8 connection(s)
I20260810 06:55:29.994004 27906 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:29.997596 28029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:30.000739 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 27906
I20260810 06:55:30.000928 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:30.001488 22574 external_mini_cluster.cc:1428] Setting key 2a0fa7e0930207790daa463116d14b83
I20260810 06:55:30.009310 28029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795: Bootstrap starting.
I20260810 06:55:30.012308 28029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:30.013329 28029 log.cc:824] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795: Log is configured to *not* fsync() on all Append() calls
Aug 10 06:55:30 dist-test-slave-f7th krb5kdc[27884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:30.016587 28029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795: No bootstrap required, opened a new log
I20260810 06:55:30.024310 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:30.003405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41055 (local address 127.22.11.190:34347)
0810 06:55:30.004174 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:30.004186 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:30.004229 (+ 43us) server_negotiation.cc:407] Connection header received
0810 06:55:30.005964 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:30.005989 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:30.006392 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:30.006768 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:30.008280 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:30.009676 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:30.010964 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:30.011626 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:30.011789 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:30.016770 (+ 4981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:30.016779 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:30.016794 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:30.016848 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:30.019955 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:30.020987 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:30.020995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:30.021001 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:30.021212 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:30.021695 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:30.021698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:30.021700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:30.022224 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:30.022451 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:30.022947 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 06:55:30.023237 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":355,"threads_started":1}
I20260810 06:55:30.025223 28029 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf3acf0394403be843667aefba795" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34347 } }
I20260810 06:55:30.025566 28029 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:30.025669 28029 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9daf3acf0394403be843667aefba795, State: Initialized, Role: FOLLOWER
I20260810 06:55:30.026211 28029 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf3acf0394403be843667aefba795" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34347 } }
I20260810 06:55:30.026463 28029 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:30.026624 28029 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:30.026811 28029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:30.028220 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:30.029312 28029 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf3acf0394403be843667aefba795" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34347 } }
I20260810 06:55:30.029786 28029 leader_election.cc:302] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9daf3acf0394403be843667aefba795; no voters:
I20260810 06:55:30.030396 28029 leader_election.cc:288] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:30.030495 28037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:30.032682 28029 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:30.032888 28037 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [term 1 LEADER]: Becoming Leader. State: Replica: d9daf3acf0394403be843667aefba795, State: Running, Role: LEADER
I20260810 06:55:30.033574 28037 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf3acf0394403be843667aefba795" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34347 } }
I20260810 06:55:30.038694 28044 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9daf3acf0394403be843667aefba795. Latest consensus state: current_term: 1 leader_uuid: "d9daf3acf0394403be843667aefba795" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf3acf0394403be843667aefba795" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34347 } } }
I20260810 06:55:30.039108 28044 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:30.039672 28040 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9daf3acf0394403be843667aefba795" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9daf3acf0394403be843667aefba795" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 34347 } } }
I20260810 06:55:30.039964 28040 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:30.047670 28057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:30.053742 28057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:30.055852 28034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:30.035030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52277 (local address 127.22.11.190:34347)
0810 06:55:30.035227 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:30.035234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:30.037178 (+ 1944us) server_negotiation.cc:407] Connection header received
0810 06:55:30.037426 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:30.037431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:30.037524 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:30.037626 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:30.039709 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:30.040994 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:30.043010 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:30.043736 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:30.043838 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:30.047282 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:30.047289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:30.047296 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:30.047359 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:30.051319 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:30.052436 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:30.052445 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:30.052449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:30.052605 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:30.053265 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:30.053269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:30.053272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:30.053707 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:30.055104 (+ 1397us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:30.055313 (+ 209us) server_negotiation.cc:299] Negotiation successful
0810 06:55:30.055510 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 06:55:30.063431 28057 catalog_manager.cc:1347] Generated new cluster ID: 5b3f2148e6284e24adb626d59ee633bb
I20260810 06:55:30.063536 28057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:30.073979 28057 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:30.075171 28057 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:30.092180 28057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9daf3acf0394403be843667aefba795: Generated new TSK 0
I20260810 06:55:30.093003 28057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:30.100925 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 27906
I20260810 06:55:30.102097 28020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:30.192567 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 27906
W20260810 06:55:30.220996 27882 connection.cc:535] client connection to 127.22.11.190:34347 recv error: Network error: failed to read from TLS socket (remote: 127.22.11.190:34347): Connection reset by peer (error 104)
2026-08-10T06:55:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2218 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:30 dist-test-slave-f7th krb5kdc[28083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:30 dist-test-slave-f7th krb5kdc[28083](info): set up 2 sockets
Aug 10 06:55:30 dist-test-slave-f7th krb5kdc[28083](info): commencing operation
krb5kdc: starting...
W20260810 06:55:31.480752 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.004s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:31 dist-test-slave-f7th krb5kdc[28083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:31.795874 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:44901
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:44901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:32.132541 28107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:32.132843 28107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:32.132954 28107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:32.140359 28107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:32.140468 28107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:32.140523 28107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:32.140584 28107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:32.140652 28107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:32.150557 28107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:44901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:44901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:32.151827 28107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:32.153223 28107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:32.165282 28123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:32.165325 28117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:32.165352 28118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:32.166101 28107 server_base.cc:1034] running on GCE node
I20260810 06:55:32.166880 28107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:32.168182 28107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:32.169386 28107 hybrid_clock.cc:648] HybridClock initialized: now 1786344932169350 us; error 24 us; skew 500 ppm
I20260810 06:55:32.169714 28107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:32 dist-test-slave-f7th krb5kdc[28083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344932, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:32.172703 28107 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:32.174049 28107 webserver.cc:460] Webserver started at http://127.22.11.190:34469/ using document root <none> and password file <none>
I20260810 06:55:32.174680 28107 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:32.174790 28107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:32.175081 28107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:32.177970 28107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "3c4c0ed642954029a8957cde130a74c7"
format_stamp: "Formatted at 2026-08-10 06:55:32 on dist-test-slave-f7th"
server_key: "ad195b2a2bc20b303d9c03c46b1b1551"
server_key_iv: "6e7d0f735d1d08f75cead3e177f0f617"
server_key_version: "encryptionkey@0"
I20260810 06:55:32.178709 28107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "3c4c0ed642954029a8957cde130a74c7"
format_stamp: "Formatted at 2026-08-10 06:55:32 on dist-test-slave-f7th"
server_key: "ad195b2a2bc20b303d9c03c46b1b1551"
server_key_iv: "6e7d0f735d1d08f75cead3e177f0f617"
server_key_version: "encryptionkey@0"
I20260810 06:55:32.183498 28107 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 06:55:32.186892 28137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:32.188081 28107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 06:55:32.188241 28107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "3c4c0ed642954029a8957cde130a74c7"
format_stamp: "Formatted at 2026-08-10 06:55:32 on dist-test-slave-f7th"
server_key: "ad195b2a2bc20b303d9c03c46b1b1551"
server_key_iv: "6e7d0f735d1d08f75cead3e177f0f617"
server_key_version: "encryptionkey@0"
I20260810 06:55:32.188410 28107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:32.212744 28107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:32.216802 28107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:32.217078 28107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:32.234476 28107 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:44901
I20260810 06:55:32.234506 28222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:44901 every 8 connection(s)
I20260810 06:55:32.235762 28107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:32.239540 28223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:32.245088 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28107
I20260810 06:55:32.245250 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:32.245685 22574 external_mini_cluster.cc:1428] Setting key 8733710001e8211a17b629ee41313f7b
I20260810 06:55:32.250531 28223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7: Bootstrap starting.
I20260810 06:55:32.254115 28223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:32.255280 28223 log.cc:824] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:32.258283 28223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7: No bootstrap required, opened a new log
I20260810 06:55:32.259063 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:32.247549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50911 (local address 127.22.11.190:44901)
0810 06:55:32.248426 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:32.248440 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:32.248480 (+ 40us) server_negotiation.cc:407] Connection header received
0810 06:55:32.250173 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:32.250199 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:32.250565 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:32.250883 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:32.251467 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:32.252820 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:32.254600 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:32.255283 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:32.255417 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:32.256781 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:32.256788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:32.256801 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:32.257304 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:32.257378 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:32.257402 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:32.257520 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:32.257792 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 06:55:32.257952 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":400,"threads_started":1}
I20260810 06:55:32.262606 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:32.267414 28223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4c0ed642954029a8957cde130a74c7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44901 } }
I20260810 06:55:32.267864 28223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:32.268026 28223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4c0ed642954029a8957cde130a74c7, State: Initialized, Role: FOLLOWER
I20260810 06:55:32.268669 28223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4c0ed642954029a8957cde130a74c7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44901 } }
I20260810 06:55:32.268970 28223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:32.269106 28223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:32.269306 28223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:32.272037 28223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4c0ed642954029a8957cde130a74c7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44901 } }
I20260810 06:55:32.272675 28223 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4c0ed642954029a8957cde130a74c7; no voters:
I20260810 06:55:32.273345 28223 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:32.273475 28241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:32.274353 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:32.267792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44143 (local address 127.22.11.190:44901)
0810 06:55:32.267965 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:32.267970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:32.268274 (+ 304us) server_negotiation.cc:407] Connection header received
0810 06:55:32.268476 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:32.268482 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:32.268556 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:32.268713 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:32.269171 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:32.270266 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:32.271537 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:32.272195 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:32.272315 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:32.273506 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:32.273512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:32.273517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:32.273538 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:32.273582 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:32.273602 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:32.273737 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:32.273953 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 06:55:32.274015 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:55:32.274714 28241 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3c4c0ed642954029a8957cde130a74c7, State: Running, Role: LEADER
I20260810 06:55:32.275225 28241 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4c0ed642954029a8957cde130a74c7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44901 } }
I20260810 06:55:32.275789 28223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:32.278244 28245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c4c0ed642954029a8957cde130a74c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4c0ed642954029a8957cde130a74c7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44901 } } }
I20260810 06:55:32.278642 28245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:32.278216 28248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4c0ed642954029a8957cde130a74c7. Latest consensus state: current_term: 1 leader_uuid: "3c4c0ed642954029a8957cde130a74c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4c0ed642954029a8957cde130a74c7" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 44901 } } }
I20260810 06:55:32.279950 28248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:32.280191 28260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:32.285297 28260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:32.294484 28260 catalog_manager.cc:1347] Generated new cluster ID: c77e1dff8c774745aede6c27510ebce6
I20260810 06:55:32.294577 28260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:32.315353 28260 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:32.316605 28260 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:32.324435 28260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c4c0ed642954029a8957cde130a74c7: Generated new TSK 0
I20260810 06:55:32.325239 28260 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:32.337975 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28107
W20260810 06:55:32.339376 28216 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50911
W20260810 06:55:32.340071 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28107: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:32.340217 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28107
W20260810 06:55:32.365147 28080 connection.cc:535] client connection to 127.22.11.190:44901 recv error: Network error: failed to read from TLS socket (remote: 127.22.11.190:44901): Connection reset by peer (error 104)
2026-08-10T06:55:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2143 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:32 dist-test-slave-f7th krb5kdc[28294](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:32 dist-test-slave-f7th krb5kdc[28294](info): set up 2 sockets
Aug 10 06:55:32 dist-test-slave-f7th krb5kdc[28294](info): commencing operation
krb5kdc: starting...
W20260810 06:55:33.593006 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.005s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:33 dist-test-slave-f7th krb5kdc[28294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:33.887282 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:35019
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:40907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:35019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:34.211539 28312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:34.211830 28312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:34.211977 28312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:34.219009 28312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:34.219125 28312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:34.219202 28312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:34.219293 28312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:34.219352 28312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:34.228785 28312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:40907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:35019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:35019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:34.229984 28312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:34.231240 28312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:34.242213 28321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:34.242303 28322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:34.242324 28328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:34.243145 28312 server_base.cc:1034] running on GCE node
I20260810 06:55:34.243871 28312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:34.245168 28312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:34.246366 28312 hybrid_clock.cc:648] HybridClock initialized: now 1786344934246324 us; error 29 us; skew 500 ppm
I20260810 06:55:34.246690 28312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:34 dist-test-slave-f7th krb5kdc[28294](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344934, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:34.249565 28312 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:34.250890 28312 webserver.cc:460] Webserver started at http://127.22.11.190:42143/ using document root <none> and password file <none>
I20260810 06:55:34.251488 28312 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:34.251621 28312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:34.251886 28312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:34.254714 28312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "20115758cfd743e7a9227f24f5eca7bf"
format_stamp: "Formatted at 2026-08-10 06:55:34 on dist-test-slave-f7th"
server_key: "2ffbdedda8320f577cdf8f04d72afb4b"
server_key_iv: "fd7ab795c4b53df6591dabd0f8b83591"
server_key_version: "encryptionkey@0"
I20260810 06:55:34.255421 28312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "20115758cfd743e7a9227f24f5eca7bf"
format_stamp: "Formatted at 2026-08-10 06:55:34 on dist-test-slave-f7th"
server_key: "2ffbdedda8320f577cdf8f04d72afb4b"
server_key_iv: "fd7ab795c4b53df6591dabd0f8b83591"
server_key_version: "encryptionkey@0"
I20260810 06:55:34.260490 28312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 06:55:34.263708 28337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:34.264971 28312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 06:55:34.265136 28312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "20115758cfd743e7a9227f24f5eca7bf"
format_stamp: "Formatted at 2026-08-10 06:55:34 on dist-test-slave-f7th"
server_key: "2ffbdedda8320f577cdf8f04d72afb4b"
server_key_iv: "fd7ab795c4b53df6591dabd0f8b83591"
server_key_version: "encryptionkey@0"
I20260810 06:55:34.265302 28312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:34.270634 28312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:34.274302 28312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:34.274559 28312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:34.290464 28312 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:35019
I20260810 06:55:34.290494 28424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:35019 every 8 connection(s)
I20260810 06:55:34.291770 28312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:34.295323 28428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:34.301211 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28312
I20260810 06:55:34.301376 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:34.301767 22574 external_mini_cluster.cc:1428] Setting key 05d1f4f78218257d56f5a52efd00d161
I20260810 06:55:34.306391 28428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf: Bootstrap starting.
I20260810 06:55:34.310207 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:34.303405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55611 (local address 127.22.11.190:35019)
0810 06:55:34.304294 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:34.304308 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:34.304355 (+ 47us) server_negotiation.cc:407] Connection header received
0810 06:55:34.306593 (+ 2238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:34.306620 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:34.307024 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:34.307416 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:34.308390 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:34.308398 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:34.308415 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:34.308435 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:34.308523 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:34.308552 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:34.308800 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:34.308916 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 06:55:34.309091 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":354,"threads_started":1}
I20260810 06:55:34.310235 28428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:34.311371 28428 log.cc:824] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:34.313867 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:34.314374 28428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf: No bootstrap required, opened a new log
I20260810 06:55:34.320925 28433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:34.318800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52657 (local address 127.22.11.190:35019)
0810 06:55:34.318926 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:34.318930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:34.319292 (+ 362us) server_negotiation.cc:407] Connection header received
0810 06:55:34.319409 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:34.319414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:34.319476 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:34.319574 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:34.320319 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:34.320324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:34.320329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:34.320349 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:34.320394 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:34.320412 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:34.320524 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:34.320570 (+ 46us) server_negotiation.cc:299] Negotiation successful
0810 06:55:34.320649 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260810 06:55:34.323316 28428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20115758cfd743e7a9227f24f5eca7bf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35019 } }
I20260810 06:55:34.323635 28428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:34.323714 28428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20115758cfd743e7a9227f24f5eca7bf, State: Initialized, Role: FOLLOWER
I20260810 06:55:34.324205 28428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20115758cfd743e7a9227f24f5eca7bf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35019 } }
I20260810 06:55:34.324416 28428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:34.324518 28428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:34.324702 28428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:34.326785 28428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20115758cfd743e7a9227f24f5eca7bf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35019 } }
I20260810 06:55:34.327225 28428 leader_election.cc:302] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20115758cfd743e7a9227f24f5eca7bf; no voters:
I20260810 06:55:34.327762 28428 leader_election.cc:288] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:34.327880 28443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:34.328696 28443 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [term 1 LEADER]: Becoming Leader. State: Replica: 20115758cfd743e7a9227f24f5eca7bf, State: Running, Role: LEADER
I20260810 06:55:34.329211 28443 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20115758cfd743e7a9227f24f5eca7bf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35019 } }
I20260810 06:55:34.329864 28428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:34.331878 28447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20115758cfd743e7a9227f24f5eca7bf. Latest consensus state: current_term: 1 leader_uuid: "20115758cfd743e7a9227f24f5eca7bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20115758cfd743e7a9227f24f5eca7bf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35019 } } }
I20260810 06:55:34.332330 28447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:34.332577 28446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20115758cfd743e7a9227f24f5eca7bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20115758cfd743e7a9227f24f5eca7bf" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 35019 } } }
I20260810 06:55:34.333005 28446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:34.334591 28453 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:34.339938 28453 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:34.348364 28453 catalog_manager.cc:1347] Generated new cluster ID: b771dc44ff0848a29949492b364afb49
I20260810 06:55:34.348470 28453 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:34.368839 28453 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:34.369230 28453 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:34.376497 28453 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20115758cfd743e7a9227f24f5eca7bf: Generated new TSK 0
I20260810 06:55:34.377311 28453 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:34.388491 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28312
W20260810 06:55:34.389570 28420 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55611
W20260810 06:55:34.390244 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28312: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:34.390389 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28312
2026-08-10T06:55:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2049 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:34 dist-test-slave-f7th krb5kdc[28483](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:34 dist-test-slave-f7th krb5kdc[28483](info): set up 2 sockets
Aug 10 06:55:34 dist-test-slave-f7th krb5kdc[28483](info): commencing operation
krb5kdc: starting...
W20260810 06:55:35.643867 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:35 dist-test-slave-f7th krb5kdc[28483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:35.937386 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:41455
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:34917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:41455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:36.263000 28509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:36.263264 28509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:36.263358 28509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:36.270191 28509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:36.270288 28509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:36.270341 28509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:36.270399 28509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:36.270442 28509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:36.279668 28509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:34917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:41455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:41455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:36.280840 28509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:36.282058 28509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:36.292971 28528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:36.293272 28519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:36.293241 28525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:36.294044 28509 server_base.cc:1034] running on GCE node
I20260810 06:55:36.294759 28509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:36.296062 28509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:36.297254 28509 hybrid_clock.cc:648] HybridClock initialized: now 1786344936297208 us; error 34 us; skew 500 ppm
I20260810 06:55:36.297569 28509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:36 dist-test-slave-f7th krb5kdc[28483](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344936, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:36.303792 28509 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:36.305202 28509 webserver.cc:460] Webserver started at http://127.22.11.190:41473/ using document root <none> and password file <none>
I20260810 06:55:36.305819 28509 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:36.305944 28509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:36.306221 28509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:36.309114 28509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "12a302939e684e58a49bafbdbaef4546"
format_stamp: "Formatted at 2026-08-10 06:55:36 on dist-test-slave-f7th"
server_key: "897de0c52e132cfba7b64b341773f98f"
server_key_iv: "8fe8bdf934b5410cc84c15738f617e3d"
server_key_version: "encryptionkey@0"
I20260810 06:55:36.309830 28509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "12a302939e684e58a49bafbdbaef4546"
format_stamp: "Formatted at 2026-08-10 06:55:36 on dist-test-slave-f7th"
server_key: "897de0c52e132cfba7b64b341773f98f"
server_key_iv: "8fe8bdf934b5410cc84c15738f617e3d"
server_key_version: "encryptionkey@0"
I20260810 06:55:36.314670 28509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:55:36.317888 28539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:36.319067 28509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260810 06:55:36.319238 28509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "12a302939e684e58a49bafbdbaef4546"
format_stamp: "Formatted at 2026-08-10 06:55:36 on dist-test-slave-f7th"
server_key: "897de0c52e132cfba7b64b341773f98f"
server_key_iv: "8fe8bdf934b5410cc84c15738f617e3d"
server_key_version: "encryptionkey@0"
I20260810 06:55:36.319433 28509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:36.340391 28509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:36.344079 28509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:36.344333 28509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:36.361645 28509 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:41455
I20260810 06:55:36.361670 28631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:41455 every 8 connection(s)
I20260810 06:55:36.362982 28509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:36.364375 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28509
I20260810 06:55:36.364531 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:36.364990 22574 external_mini_cluster.cc:1428] Setting key a357caef043906d18d9c611e3d59d3a5
I20260810 06:55:36.366931 28632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 06:55:36 dist-test-slave-f7th krb5kdc[28483](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:36.379352 28632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546: Bootstrap starting.
I20260810 06:55:36.382277 28632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:36.383270 28632 log.cc:824] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:36.385833 28632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546: No bootstrap required, opened a new log
I20260810 06:55:36.386852 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:36.366642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53341 (local address 127.22.11.190:41455)
0810 06:55:36.367402 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:36.367413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:36.367452 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:55:36.369133 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:36.369155 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:36.369522 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:36.369836 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:36.371582 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:36.372965 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:36.374675 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:36.375312 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:36.375446 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:36.379999 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:36.380006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:36.380019 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:36.380052 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:36.382952 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:36.383606 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:36.383613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:36.383618 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:36.383792 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:36.384176 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:36.384180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:36.384182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:36.384736 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:36.385003 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:36.385436 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 06:55:36.385808 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":566,"thread_start_us":357,"threads_started":1}
I20260810 06:55:36.390218 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:36.393375 28632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12a302939e684e58a49bafbdbaef4546" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41455 } }
I20260810 06:55:36.393684 28632 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:36.393774 28632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12a302939e684e58a49bafbdbaef4546, State: Initialized, Role: FOLLOWER
I20260810 06:55:36.394419 28632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12a302939e684e58a49bafbdbaef4546" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41455 } }
I20260810 06:55:36.394691 28632 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:36.394826 28632 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:36.394994 28632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:36.397095 28632 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12a302939e684e58a49bafbdbaef4546" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41455 } }
I20260810 06:55:36.397545 28632 leader_election.cc:302] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12a302939e684e58a49bafbdbaef4546; no voters:
I20260810 06:55:36.398157 28632 leader_election.cc:288] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:36.398234 28653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:36.400259 28632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:36.405056 28653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [term 1 LEADER]: Becoming Leader. State: Replica: 12a302939e684e58a49bafbdbaef4546, State: Running, Role: LEADER
I20260810 06:55:36.405707 28653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12a302939e684e58a49bafbdbaef4546" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41455 } }
I20260810 06:55:36.407176 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:36.394290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32867 (local address 127.22.11.190:41455)
0810 06:55:36.394431 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:36.394436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:36.394902 (+ 466us) server_negotiation.cc:407] Connection header received
0810 06:55:36.395066 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:36.395072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:36.395140 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:36.395222 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:36.396677 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:36.397411 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:36.398809 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:36.399429 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:36.399511 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:36.401991 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:36.401997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:36.402000 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:36.402027 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:36.404608 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:36.405235 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:36.405241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:36.405245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:36.405404 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:36.405733 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:36.405738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:36.405742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:36.406035 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:36.406290 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:36.406609 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 06:55:36.406800 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260810 06:55:36.408812 28654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12a302939e684e58a49bafbdbaef4546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12a302939e684e58a49bafbdbaef4546" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41455 } } }
I20260810 06:55:36.409219 28654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:36.409893 28657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12a302939e684e58a49bafbdbaef4546. Latest consensus state: current_term: 1 leader_uuid: "12a302939e684e58a49bafbdbaef4546" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12a302939e684e58a49bafbdbaef4546" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 41455 } } }
I20260810 06:55:36.410248 28657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546 [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:36.410487 28664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:36.415910 28664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:36.425910 28664 catalog_manager.cc:1347] Generated new cluster ID: ddc288957629400c8d8ce2fcf0adfd79
I20260810 06:55:36.426017 28664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:36.448284 28664 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:36.450048 28664 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:36.462975 28664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12a302939e684e58a49bafbdbaef4546: Generated new TSK 0
I20260810 06:55:36.463945 28664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:36.471159 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28509
I20260810 06:55:36.472203 28620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:36.561611 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28509
2026-08-10T06:55:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2169 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:36 dist-test-slave-f7th krb5kdc[28686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:36 dist-test-slave-f7th krb5kdc[28686](info): set up 2 sockets
Aug 10 06:55:36 dist-test-slave-f7th krb5kdc[28686](info): commencing operation
krb5kdc: starting...
W20260810 06:55:37.814102 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.001s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:38 dist-test-slave-f7th krb5kdc[28686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:38.104005 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45645
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:45645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:38.436444 28703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:38.436735 28703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:38.436836 28703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:38.443822 28703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:38.443924 28703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:38.443977 28703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:38.444036 28703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:38.444077 28703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:38.453611 28703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:45645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:45645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:38.454808 28703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:38.456048 28703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:38.467334 28715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:38.467336 28713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:38.467280 28710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:38.468068 28703 server_base.cc:1034] running on GCE node
I20260810 06:55:38.468839 28703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:38.470103 28703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:38.471303 28703 hybrid_clock.cc:648] HybridClock initialized: now 1786344938471259 us; error 35 us; skew 500 ppm
I20260810 06:55:38.471626 28703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:38 dist-test-slave-f7th krb5kdc[28686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344938, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:38.474573 28703 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:38.475932 28703 webserver.cc:460] Webserver started at http://127.22.11.190:36457/ using document root <none> and password file <none>
I20260810 06:55:38.476536 28703 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:38.476679 28703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:38.476950 28703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:38.479739 28703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "83caf9ab42a5465e8d47eb68c965ed5e"
format_stamp: "Formatted at 2026-08-10 06:55:38 on dist-test-slave-f7th"
server_key: "b01757a7659b842629e6e29889889c04"
server_key_iv: "2bc29db99812995bc6299ef54f827d53"
server_key_version: "encryptionkey@0"
I20260810 06:55:38.480453 28703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "83caf9ab42a5465e8d47eb68c965ed5e"
format_stamp: "Formatted at 2026-08-10 06:55:38 on dist-test-slave-f7th"
server_key: "b01757a7659b842629e6e29889889c04"
server_key_iv: "2bc29db99812995bc6299ef54f827d53"
server_key_version: "encryptionkey@0"
I20260810 06:55:38.485333 28703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 06:55:38.488596 28725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:38.489902 28703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 06:55:38.490096 28703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "83caf9ab42a5465e8d47eb68c965ed5e"
format_stamp: "Formatted at 2026-08-10 06:55:38 on dist-test-slave-f7th"
server_key: "b01757a7659b842629e6e29889889c04"
server_key_iv: "2bc29db99812995bc6299ef54f827d53"
server_key_version: "encryptionkey@0"
I20260810 06:55:38.490259 28703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:38.508791 28703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:38.512411 28703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:38.512696 28703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:38.531466 28703 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:45645
I20260810 06:55:38.531486 28818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:45645 every 8 connection(s)
I20260810 06:55:38.532780 28703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:38.533010 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28703
I20260810 06:55:38.533165 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:38.533658 22574 external_mini_cluster.cc:1428] Setting key 9a3d7d8d4fb1ae0c03ccc8b2a3a2b62e
I20260810 06:55:38.536955 28819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:38.546252 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:38.535337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39075 (local address 127.22.11.190:45645)
0810 06:55:38.536133 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:38.536144 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:38.536183 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:55:38.537848 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:38.537870 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:38.538218 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:38.538522 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:38.539091 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:38.540421 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:38.542155 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:38.542814 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:38.542933 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:38.544120 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:38.544127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:38.544140 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:38.544667 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:38.544750 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:38.544779 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:38.544859 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:38.545085 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 06:55:38.545308 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":324,"threads_started":1}
I20260810 06:55:38.549101 28819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e: Bootstrap starting.
I20260810 06:55:38.549817 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:38.552014 28819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:38.553210 28819 log.cc:824] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:38.556408 28819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e: No bootstrap required, opened a new log
I20260810 06:55:38.560053 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:38.553873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36895 (local address 127.22.11.190:45645)
0810 06:55:38.554034 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:38.554040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:38.554478 (+ 438us) server_negotiation.cc:407] Connection header received
0810 06:55:38.554673 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:38.554678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:38.554745 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:38.554857 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:38.555278 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:38.556292 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:38.557612 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:38.558298 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:38.558432 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:38.559341 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:38.559347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:38.559352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:38.559372 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:38.559419 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:38.559439 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:38.559533 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:38.559615 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 06:55:38.559701 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 06:55:38.564443 28819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45645 } }
I20260810 06:55:38.564821 28819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:38.564934 28819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83caf9ab42a5465e8d47eb68c965ed5e, State: Initialized, Role: FOLLOWER
I20260810 06:55:38.565517 28819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45645 } }
I20260810 06:55:38.565740 28819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:38.565871 28819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:38.566030 28819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:38.568069 28819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45645 } }
I20260810 06:55:38.568538 28819 leader_election.cc:302] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83caf9ab42a5465e8d47eb68c965ed5e; no voters:
I20260810 06:55:38.569106 28819 leader_election.cc:288] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:38.569216 28839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:38.569864 28839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [term 1 LEADER]: Becoming Leader. State: Replica: 83caf9ab42a5465e8d47eb68c965ed5e, State: Running, Role: LEADER
I20260810 06:55:38.570405 28839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45645 } }
I20260810 06:55:38.571055 28819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:38.573239 28840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45645 } } }
I20260810 06:55:38.573272 28841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83caf9ab42a5465e8d47eb68c965ed5e. Latest consensus state: current_term: 1 leader_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83caf9ab42a5465e8d47eb68c965ed5e" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 45645 } } }
I20260810 06:55:38.573606 28840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:38.573617 28841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:38.575064 28845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:38.581306 28845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:38.589519 28845 catalog_manager.cc:1347] Generated new cluster ID: a2c1613a95444dd49a4eb2ad6c7bd80a
I20260810 06:55:38.589622 28845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:38.610457 28845 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:38.611748 28845 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:38.630606 28845 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83caf9ab42a5465e8d47eb68c965ed5e: Generated new TSK 0
I20260810 06:55:38.631738 28845 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:38.639721 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28703
W20260810 06:55:38.640820 28811 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39075
W20260810 06:55:38.641494 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28703: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:38.641626 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28703
2026-08-10T06:55:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2088 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:38 dist-test-slave-f7th krb5kdc[28871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:38 dist-test-slave-f7th krb5kdc[28871](info): set up 2 sockets
Aug 10 06:55:38 dist-test-slave-f7th krb5kdc[28871](info): commencing operation
krb5kdc: starting...
W20260810 06:55:39.913343 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:40 dist-test-slave-f7th krb5kdc[28871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:40.211444 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:33369
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:35667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:33369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:40.542043 28899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:40.542308 28899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:40.542439 28899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:40.549422 28899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:40.549535 28899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:40.549610 28899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:40.549691 28899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:40.549748 28899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:40.559077 28899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:35667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:33369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:33369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:40.560225 28899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:40.561498 28899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:40.573904 28911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:40.573984 28913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:40.573930 28910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:40.575767 28899 server_base.cc:1034] running on GCE node
I20260810 06:55:40.576704 28899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:40.578394 28899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:40.579635 28899 hybrid_clock.cc:648] HybridClock initialized: now 1786344940579544 us; error 73 us; skew 500 ppm
I20260810 06:55:40.580041 28899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:40 dist-test-slave-f7th krb5kdc[28871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344940, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:40.583640 28899 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:40.585016 28899 webserver.cc:460] Webserver started at http://127.22.11.190:44405/ using document root <none> and password file <none>
I20260810 06:55:40.585646 28899 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:40.585752 28899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:40.586031 28899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:40.588968 28899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "23b23cbb24454174b40cd8f3fa3ac69f"
format_stamp: "Formatted at 2026-08-10 06:55:40 on dist-test-slave-f7th"
server_key: "43053322f4a9c1269b1f77e083635703"
server_key_iv: "4434b2d28c83184ff77500850de24efa"
server_key_version: "encryptionkey@0"
I20260810 06:55:40.589676 28899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "23b23cbb24454174b40cd8f3fa3ac69f"
format_stamp: "Formatted at 2026-08-10 06:55:40 on dist-test-slave-f7th"
server_key: "43053322f4a9c1269b1f77e083635703"
server_key_iv: "4434b2d28c83184ff77500850de24efa"
server_key_version: "encryptionkey@0"
I20260810 06:55:40.594769 28899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:55:40.598204 28923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:40.599529 28899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 06:55:40.599687 28899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "23b23cbb24454174b40cd8f3fa3ac69f"
format_stamp: "Formatted at 2026-08-10 06:55:40 on dist-test-slave-f7th"
server_key: "43053322f4a9c1269b1f77e083635703"
server_key_iv: "4434b2d28c83184ff77500850de24efa"
server_key_version: "encryptionkey@0"
I20260810 06:55:40.599851 28899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:40.605360 28899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:40.616711 28899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:40.617048 28899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:40.635881 29009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:33369 every 8 connection(s)
I20260810 06:55:40.635881 28899 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:33369
I20260810 06:55:40.637699 28899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:40.638535 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 28899
I20260810 06:55:40.638724 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:40.639256 22574 external_mini_cluster.cc:1428] Setting key 692f1908de83eb0cb1355dcaa9497d29
I20260810 06:55:40.642309 29012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:40.648440 29015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:40.641142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52831 (local address 127.22.11.190:33369)
0810 06:55:40.641996 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:40.642012 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:40.642064 (+ 52us) server_negotiation.cc:407] Connection header received
0810 06:55:40.644401 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:40.644430 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:40.644951 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:40.645406 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:40.646395 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:40.646404 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:40.646425 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:40.646443 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:40.646534 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:40.646564 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:40.646788 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:40.647002 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 06:55:40.647194 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":338,"threads_started":1}
I20260810 06:55:40.652680 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:40.655028 29012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f: Bootstrap starting.
I20260810 06:55:40.658764 29012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:40.659776 29012 log.cc:824] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:40.662585 29015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:40.657989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59391 (local address 127.22.11.190:33369)
0810 06:55:40.658176 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:40.658181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:40.660511 (+ 2330us) server_negotiation.cc:407] Connection header received
0810 06:55:40.660699 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:40.660704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:40.660767 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:40.660897 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:40.661863 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:40.661870 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:40.661875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 06:55:40.661895 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:40.661949 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 06:55:40.661969 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:40.662088 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:40.662223 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:55:40.662283 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:55:40.662917 29012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f: No bootstrap required, opened a new log
I20260810 06:55:40.670421 29012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33369 } }
I20260810 06:55:40.670754 29012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 06:55:40.670841 29012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23b23cbb24454174b40cd8f3fa3ac69f, State: Initialized, Role: FOLLOWER
I20260810 06:55:40.671351 29012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33369 } }
I20260810 06:55:40.671559 29012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 06:55:40.671682 29012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 06:55:40.671833 29012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:40.673969 29012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33369 } }
I20260810 06:55:40.674425 29012 leader_election.cc:302] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23b23cbb24454174b40cd8f3fa3ac69f; no voters:
I20260810 06:55:40.675020 29012 leader_election.cc:288] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 06:55:40.675133 29033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:40.677242 29012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:40.681905 29033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [term 1 LEADER]: Becoming Leader. State: Replica: 23b23cbb24454174b40cd8f3fa3ac69f, State: Running, Role: LEADER
I20260810 06:55:40.682590 29033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33369 } }
I20260810 06:55:40.685482 29034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33369 } } }
I20260810 06:55:40.685681 29035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23b23cbb24454174b40cd8f3fa3ac69f. Latest consensus state: current_term: 1 leader_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23b23cbb24454174b40cd8f3fa3ac69f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 33369 } } }
I20260810 06:55:40.685899 29034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:40.685926 29035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:40.688712 29044 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:40.694303 29044 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:40.703552 29044 catalog_manager.cc:1347] Generated new cluster ID: 3b44619da65248eea47e72d7a7f95bd5
I20260810 06:55:40.703641 29044 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:40.734751 29044 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:40.735260 29044 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:40.747176 29044 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23b23cbb24454174b40cd8f3fa3ac69f: Generated new TSK 0
I20260810 06:55:40.748181 29044 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:40.753777 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28899
W20260810 06:55:40.754791 29004 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52831
W20260810 06:55:40.755419 22574 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 28899: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 06:55:40.755566 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 28899
2026-08-10T06:55:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2110 ms)
[----------] 18 tests from AuthTokenIssuingTest (37975 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:40 dist-test-slave-f7th krb5kdc[29069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:40 dist-test-slave-f7th krb5kdc[29069](info): set up 2 sockets
Aug 10 06:55:40 dist-test-slave-f7th krb5kdc[29069](info): commencing operation
krb5kdc: starting...
W20260810 06:55:42.019536 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:42 dist-test-slave-f7th krb5kdc[29069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:42.311264 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39773
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:39773,127.22.11.189:34111,127.22.11.188:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:42.641062 29091 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:55:42.641368 29091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:42.641469 29091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:42.641585 29091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:42.648758 29091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:42.648849 29091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:42.648942 29091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:42.649009 29091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:42.649093 29091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:42.658843 29091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:39773,127.22.11.189:34111,127.22.11.188:42825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:39773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:42.660118 29091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:42.661478 29091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:42.672771 29099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:42.672673 29100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:42.672816 29104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:42.673578 29091 server_base.cc:1034] running on GCE node
I20260810 06:55:42.674291 29091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:42.675535 29091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:42.676730 29091 hybrid_clock.cc:648] HybridClock initialized: now 1786344942676677 us; error 24 us; skew 500 ppm
I20260810 06:55:42.677033 29091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:42 dist-test-slave-f7th krb5kdc[29069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:42.682861 29091 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:42.684269 29091 webserver.cc:460] Webserver started at http://127.22.11.190:33827/ using document root <none> and password file <none>
I20260810 06:55:42.684962 29091 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:42.685065 29091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:42.685367 29091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:42.688380 29091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "f3b2cf6f15044dd98c032bede094dedd"
format_stamp: "Formatted at 2026-08-10 06:55:42 on dist-test-slave-f7th"
server_key: "197a3d98277b5b8f7c7d6b1c40b55cf0"
server_key_iv: "2fd382b8b81232115f755f739e490399"
server_key_version: "encryptionkey@0"
I20260810 06:55:42.689208 29091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "f3b2cf6f15044dd98c032bede094dedd"
format_stamp: "Formatted at 2026-08-10 06:55:42 on dist-test-slave-f7th"
server_key: "197a3d98277b5b8f7c7d6b1c40b55cf0"
server_key_iv: "2fd382b8b81232115f755f739e490399"
server_key_version: "encryptionkey@0"
I20260810 06:55:42.694129 29091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 06:55:42.697409 29118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:42.698645 29091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 06:55:42.698810 29091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "f3b2cf6f15044dd98c032bede094dedd"
format_stamp: "Formatted at 2026-08-10 06:55:42 on dist-test-slave-f7th"
server_key: "197a3d98277b5b8f7c7d6b1c40b55cf0"
server_key_iv: "2fd382b8b81232115f755f739e490399"
server_key_version: "encryptionkey@0"
I20260810 06:55:42.699011 29091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:42.729627 29091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:42.733299 29091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:42.733548 29091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:42.750488 29091 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:39773
I20260810 06:55:42.750525 29209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:39773 every 8 connection(s)
I20260810 06:55:42.751699 29091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:42.755200 29210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:42.756458 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29091
I20260810 06:55:42.756675 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:42.757347 22574 external_mini_cluster.cc:1428] Setting key 335017b20d5171a5565741366a9f76da
I20260810 06:55:42.761528 29210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:42 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:42.784355 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.764689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39773 (local address 127.0.0.1:33884)
0810 06:55:42.765543 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:42.765599 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:42.767883 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:42.768713 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:42.768719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:42.769413 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:42.769852 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:42.769876 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:42.771436 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:42.771442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:42.772735 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:42.772748 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:42.773926 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:42.773931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:42.774060 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:42.774647 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:42.774669 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:42.776991 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:42.780675 (+ 3684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:42.780685 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:42.780693 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:42.781113 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:42.781550 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:42.781553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:42.781555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:42.781707 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:42.782594 (+ 887us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:42.782600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:42.782786 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 06:55:42.782990 (+ 204us) client_negotiation.cc:240] Negotiation successful
0810 06:55:42.783357 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":361,"threads_started":1}
I20260810 06:55:42.784387 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.764689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.22.11.190:39773)
0810 06:55:42.766394 (+ 1705us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:42.766400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:42.766444 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:55:42.768238 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:42.768244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:42.768466 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:42.768645 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:42.770084 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:42.771278 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:42.772903 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:42.773700 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:42.773853 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:42.777225 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:42.777235 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:42.777247 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:42.777279 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:42.780525 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:42.781256 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:42.781264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:42.781267 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:42.781446 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:42.781815 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:42.781820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:42.781823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:42.782450 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:42.782635 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:42.783169 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 06:55:42.783539 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":1517,"spinlock_wait_cycles":16256,"thread_start_us":1294,"threads_started":1}
I20260810 06:55:42.787144 29210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:42.788219 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.787856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32850)
0810 06:55:42.788017 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:42.788119 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 06:55:42.788534 29125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
I20260810 06:55:42.790107 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.789738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32854)
0810 06:55:42.789899 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:42.789978 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 06:55:42.790320 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:34111: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.11.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab.
Entry for principal kudu/127.22.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab.
I20260810 06:55:42.827047 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:34111
--webserver_interface=127.22.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab
--principal=kudu/127.22.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:39773,127.22.11.189:34111,127.22.11.188:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:55:42.839629 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } attempt: 1
I20260810 06:55:42.840819 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.840331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32858)
0810 06:55:42.840516 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:42.840655 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:55:42.841741 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.841436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32874)
0810 06:55:42.841559 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:42.841625 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:55:42.841925 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:34111: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
I20260810 06:55:42.910153 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } attempt: 2
I20260810 06:55:42.911345 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.910863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32880)
0810 06:55:42.911111 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:42.911204 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:55:42.912369 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:42.911993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32884)
0810 06:55:42.912146 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:42.912238 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:55:42.912561 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:34111: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
I20260810 06:55:43.003823 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } attempt: 3
I20260810 06:55:43.004892 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.004440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32896)
0810 06:55:43.004661 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.004756 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 06:55:43.005940 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.005605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32906)
0810 06:55:43.005733 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.005802 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260810 06:55:43.006122 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:34111: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
I20260810 06:55:43.149408 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } attempt: 4
I20260810 06:55:43.150672 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.150098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32914)
0810 06:55:43.150441 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.150530 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260810 06:55:43.151672 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.151304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32930)
0810 06:55:43.151460 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.151545 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 06:55:43.151888 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:34111: Network error: Client connection negotiation failed: client connection to 127.22.11.189:34111: connect: Connection refused (error 111)
W20260810 06:55:43.159873 29218 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:55:43.160215 29218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:43.160313 29218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:43.160434 29218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:43.167776 29218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:43.167869 29218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:43.167959 29218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:43.168035 29218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:43.168123 29218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:43.178248 29218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:39773,127.22.11.189:34111,127.22.11.188:42825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab
--principal=kudu/127.22.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:34111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:43.179517 29218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:43.180858 29218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:43.191968 29225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:43.191951 29226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:43.191973 29231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:43.193429 29218 server_base.cc:1034] running on GCE node
I20260810 06:55:43.194216 29218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:43.195601 29218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:43.196810 29218 hybrid_clock.cc:648] HybridClock initialized: now 1786344943196747 us; error 37 us; skew 500 ppm
I20260810 06:55:43.197130 29218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:43.200119 29218 init.cc:377] Logged in from keytab as kudu/127.22.11.189@KRBTEST.COM (short username kudu)
I20260810 06:55:43.201488 29218 webserver.cc:460] Webserver started at http://127.22.11.189:45143/ using document root <none> and password file <none>
I20260810 06:55:43.202230 29218 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:43.202322 29218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:43.202618 29218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:43.205587 29218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data/instance:
uuid: "8afd572643a342879e1bd394a4a3ad73"
format_stamp: "Formatted at 2026-08-10 06:55:43 on dist-test-slave-f7th"
server_key: "356b38ee7c67b3583d2acc4b1a8aaf3e"
server_key_iv: "04961d7ec47e694892101177747ad6c7"
server_key_version: "encryptionkey@0"
I20260810 06:55:43.206327 29218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/wal/instance:
uuid: "8afd572643a342879e1bd394a4a3ad73"
format_stamp: "Formatted at 2026-08-10 06:55:43 on dist-test-slave-f7th"
server_key: "356b38ee7c67b3583d2acc4b1a8aaf3e"
server_key_iv: "04961d7ec47e694892101177747ad6c7"
server_key_version: "encryptionkey@0"
I20260810 06:55:43.211207 29218 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 06:55:43.214437 29242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:43.215649 29218 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 06:55:43.215799 29218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/wal
uuid: "8afd572643a342879e1bd394a4a3ad73"
format_stamp: "Formatted at 2026-08-10 06:55:43 on dist-test-slave-f7th"
server_key: "356b38ee7c67b3583d2acc4b1a8aaf3e"
server_key_iv: "04961d7ec47e694892101177747ad6c7"
server_key_version: "encryptionkey@0"
I20260810 06:55:43.215981 29218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:43.241591 29218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:43.245333 29218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:43.245577 29218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:43.265914 29218 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.189:34111
I20260810 06:55:43.265935 29328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.189:34111 every 8 connection(s)
I20260810 06:55:43.267274 29218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
I20260810 06:55:43.270958 29330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:43.275023 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29218
I20260810 06:55:43.275213 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-1/wal/instance
I20260810 06:55:43.275741 22574 external_mini_cluster.cc:1428] Setting key 1f4112c4564d99721700e66130a08514
I20260810 06:55:43.278291 29330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:43.299041 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.281298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.22.11.190:39773)
0810 06:55:43.281508 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.281514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.282584 (+ 1070us) server_negotiation.cc:407] Connection header received
0810 06:55:43.284840 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.284845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.284919 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.285006 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.286812 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.287499 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.289622 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.290055 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.290150 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.293557 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.293562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.293565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.293598 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.296092 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.296827 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.296833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.296835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.296953 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.297372 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.297376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.297378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.297663 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.297808 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.298482 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.298713 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:55:43.299981 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.281659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39773 (local address 127.0.0.1:33900)
0810 06:55:43.282392 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.282441 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.284551 (+ 2110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.285244 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.285254 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.285924 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.286664 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.286687 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.287659 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.287664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.289472 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.289487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.290268 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.290274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.290529 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.291212 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.291234 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.293336 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:43.296221 (+ 2885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.296230 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.296246 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.296707 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.297074 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.297079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.297083 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.297260 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.297786 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:43.297794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:43.298126 (+ 332us) client_negotiation.cc:769] Sending connection context
0810 06:55:43.298354 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 06:55:43.298881 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":283,"threads_started":1}
I20260810 06:55:43.301744 29330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.22.11.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.188@KRBTEST.COM" created.
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:55:43.316094 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.302531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32946)
0810 06:55:43.302704 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.302724 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.302818 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.303881 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.303885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.304339 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.304508 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.304516 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.305857 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.305863 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.306685 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.306696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.307474 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.307479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.307668 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.308111 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.308126 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.310979 (+ 2853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:43.313834 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.313839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.313843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.314183 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.314524 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.314530 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.314533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.314644 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.315180 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:43.315184 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:43.315337 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:55:43.315454 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 06:55:43.315713 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:55:43.316172 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.302790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32946 (local address 127.22.11.189:34111)
0810 06:55:43.303510 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.303515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.303540 (+ 25us) server_negotiation.cc:407] Connection header received
0810 06:55:43.303582 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.303587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.303743 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.303871 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.304681 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.305733 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.306807 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.307346 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.307453 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.311165 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.311175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.311187 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.311214 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.313717 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.314296 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.314302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.314304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.314408 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.314738 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.314743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.314746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.315066 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.315262 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.315602 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.315804 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":382,"threads_started":1}
I20260810 06:55:43.318625 29330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:43.319774 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.319304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36124)
0810 06:55:43.319468 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.319606 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 06:55:43.320083 29247 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
I20260810 06:55:43.321589 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.321219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36128)
0810 06:55:43.321380 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.321484 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260810 06:55:43.321779 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab.
Entry for principal kudu/127.22.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab.
I20260810 06:55:43.344982 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:42825
--webserver_interface=127.22.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab
--principal=kudu/127.22.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:42157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:39773,127.22.11.189:34111,127.22.11.188:42825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:55:43.371075 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 1
I20260810 06:55:43.372197 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.371732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36132)
0810 06:55:43.371930 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.372046 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 06:55:43.373281 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.372975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36136)
0810 06:55:43.373094 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.373178 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:55:43.373481 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
I20260810 06:55:43.441677 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 2
I20260810 06:55:43.442795 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.442368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36146)
0810 06:55:43.442547 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.442638 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 06:55:43.443711 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.443425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36148)
0810 06:55:43.443539 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.443603 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260810 06:55:43.443926 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
I20260810 06:55:43.451119 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } attempt: 5
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:55:43.463598 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.451700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32960)
0810 06:55:43.451885 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.451904 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.452045 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.452382 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.452387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.452833 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.453021 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.453031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.453965 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.453971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.454835 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.454847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.455589 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.455595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.455795 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.456347 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.456364 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.458548 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:43.461358 (+ 2810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.461363 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.461368 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.461718 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.462054 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.462059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.462063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.462195 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.462692 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:43.462697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:43.462851 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 06:55:43.462970 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 06:55:43.463245 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:55:43.463600 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.451738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32960 (local address 127.22.11.189:34111)
0810 06:55:43.451904 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.451909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.451976 (+ 67us) server_negotiation.cc:407] Connection header received
0810 06:55:43.452162 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.452168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.452245 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.452328 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.453160 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.453839 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.454988 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.455442 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.455551 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.458719 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.458723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.458727 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.458759 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.461237 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.461826 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.461831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.461833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.461949 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.462292 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.462297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.462300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.462580 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.462713 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.463085 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.463288 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 06:55:43.464789 29210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:43.465770 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.465423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36150)
0810 06:55:43.465568 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.465639 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 06:55:43.466624 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.466325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36164)
0810 06:55:43.466458 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.466532 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:55:43.466813 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
I20260810 06:55:43.518034 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 1
I20260810 06:55:43.519101 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.518637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36178)
0810 06:55:43.518822 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.518918 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:55:43.520212 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.519840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36188)
0810 06:55:43.520030 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.520100 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:55:43.520498 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
I20260810 06:55:43.535113 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 3
I20260810 06:55:43.536113 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.535701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36192)
0810 06:55:43.535878 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.535961 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 06:55:43.537196 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.536771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36194)
0810 06:55:43.537022 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.537095 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260810 06:55:43.537406 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
I20260810 06:55:43.588775 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 2
I20260810 06:55:43.589946 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.589479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36210)
0810 06:55:43.589698 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.589783 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 06:55:43.590945 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.590647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36224)
0810 06:55:43.590780 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.590848 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 06:55:43.591149 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
W20260810 06:55:43.678877 29341 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:55:43.679198 29341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:43.679296 29341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:43.679407 29341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 06:55:43.680591 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 4
I20260810 06:55:43.681730 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.681273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36234)
0810 06:55:43.681481 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.681582 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 06:55:43.682665 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.682349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36238)
0810 06:55:43.682462 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.682543 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 06:55:43.682871 29330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
W20260810 06:55:43.687173 29341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:43.687252 29341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:43.687337 29341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:43.687397 29341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:43.687481 29341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:43.697237 29341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:42157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:39773,127.22.11.189:34111,127.22.11.188:42825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab
--principal=kudu/127.22.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:42825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:43.697330 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 3
I20260810 06:55:43.698376 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.697947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36242)
0810 06:55:43.698150 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.698231 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 06:55:43.698719 29341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:43.699383 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.698982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36254)
0810 06:55:43.699195 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.699281 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260810 06:55:43.699587 29210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:42825: Network error: Client connection negotiation failed: client connection to 127.22.11.188:42825: connect: Connection refused (error 111)
I20260810 06:55:43.700274 29341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:43.711534 29348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:43.711557 29351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:43.711499 29349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:43.712348 29341 server_base.cc:1034] running on GCE node
I20260810 06:55:43.713110 29341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:43.714488 29341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:43.715693 29341 hybrid_clock.cc:648] HybridClock initialized: now 1786344943715622 us; error 46 us; skew 500 ppm
I20260810 06:55:43.716001 29341 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:43.719034 29341 init.cc:377] Logged in from keytab as kudu/127.22.11.188@KRBTEST.COM (short username kudu)
I20260810 06:55:43.720383 29341 webserver.cc:460] Webserver started at http://127.22.11.188:45283/ using document root <none> and password file <none>
I20260810 06:55:43.721058 29341 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:43.721161 29341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:43.721453 29341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:43.724310 29341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data/instance:
uuid: "d155a3058f1a4288aaeee95ffb52ebd6"
format_stamp: "Formatted at 2026-08-10 06:55:43 on dist-test-slave-f7th"
server_key: "6a565bb4bd3bee437064727771c11393"
server_key_iv: "7403bf1e7f0130abb2b0bb6ffa9359b0"
server_key_version: "encryptionkey@0"
I20260810 06:55:43.725071 29341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/wal/instance:
uuid: "d155a3058f1a4288aaeee95ffb52ebd6"
format_stamp: "Formatted at 2026-08-10 06:55:43 on dist-test-slave-f7th"
server_key: "6a565bb4bd3bee437064727771c11393"
server_key_iv: "7403bf1e7f0130abb2b0bb6ffa9359b0"
server_key_version: "encryptionkey@0"
I20260810 06:55:43.729817 29341 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 06:55:43.732986 29361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:43.734223 29341 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:55:43.734380 29341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/wal
uuid: "d155a3058f1a4288aaeee95ffb52ebd6"
format_stamp: "Formatted at 2026-08-10 06:55:43 on dist-test-slave-f7th"
server_key: "6a565bb4bd3bee437064727771c11393"
server_key_iv: "7403bf1e7f0130abb2b0bb6ffa9359b0"
server_key_version: "encryptionkey@0"
I20260810 06:55:43.734570 29341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:43.755199 29341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:43.758857 29341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:43.759086 29341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:43.776692 29341 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.188:42825
I20260810 06:55:43.776714 29458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.188:42825 every 8 connection(s)
I20260810 06:55:43.778007 29341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
I20260810 06:55:43.781620 29459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:43.785267 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29341
I20260810 06:55:43.785451 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786344580063516-22574-0/minicluster-data/master-2/wal/instance
I20260810 06:55:43.785959 22574 external_mini_cluster.cc:1428] Setting key 407c719e9711c4695a4e585d5beb39b9
I20260810 06:55:43.788931 29459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:43.807507 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.792684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33908 (local address 127.22.11.190:39773)
0810 06:55:43.792888 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.792894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.793378 (+ 484us) server_negotiation.cc:407] Connection header received
0810 06:55:43.793560 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.793565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.793644 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.793750 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.794935 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.795640 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.797200 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.797635 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.797725 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.801447 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.801453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.801458 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.801498 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.804483 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.805284 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.805288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.805293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.805454 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.805864 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.805868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.805871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.806215 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.806368 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.806948 (+ 580us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.807181 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 06:55:43.808506 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.787858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36268 (local address 127.22.11.188:42825)
0810 06:55:43.788531 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.788544 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.788583 (+ 39us) server_negotiation.cc:407] Connection header received
0810 06:55:43.790134 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.790155 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.790522 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.790827 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.792495 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.794197 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.795599 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.796323 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.796443 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.801121 (+ 4678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.801128 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.801139 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.801167 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.804483 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.805283 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.805290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.805293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.805487 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.805863 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.805868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.805871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.806487 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.806711 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.807100 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.807499 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":489,"thread_start_us":297,"threads_started":1}
I20260810 06:55:43.808548 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.792569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:39773 (local address 127.0.0.1:33908)
0810 06:55:43.793235 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.793280 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.793435 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.793791 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.793795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.794458 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.794776 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.794789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.795786 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.795790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.797071 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.797083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.797847 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.797855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.798031 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.798600 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.798630 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.801263 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:43.804632 (+ 3369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.804644 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.804655 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.805184 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.805564 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.805569 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.805572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.805767 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.806358 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:43.806364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:43.806552 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 06:55:43.806826 (+ 274us) client_negotiation.cc:240] Negotiation successful
0810 06:55:43.807358 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"spinlock_wait_cycles":13056,"thread_start_us":306,"threads_started":1}
I20260810 06:55:43.810426 29459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:43.823002 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.811177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:34111 (local address 127.0.0.1:32964)
0810 06:55:43.811482 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.811515 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.811663 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.811954 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.811958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.812294 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.812429 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.812437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.813414 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.813419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.814264 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.814275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.814931 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.814937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.815098 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.815565 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.815580 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.817851 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:43.820854 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.820860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.820864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.821197 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.821536 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.821541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.821545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.821676 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.822162 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:43.822166 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:43.822301 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 06:55:43.822415 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 06:55:43.822669 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209}
I20260810 06:55:43.823083 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.811171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32964 (local address 127.22.11.189:34111)
0810 06:55:43.811356 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.811362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.811579 (+ 217us) server_negotiation.cc:407] Connection header received
0810 06:55:43.811780 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.811784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.811845 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.811923 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.812577 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.813287 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.814394 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.814803 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.814908 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.818023 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.818028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.818031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.818063 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.820733 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.821304 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.821309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.821313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.821435 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.821775 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.821780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.821783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.822041 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.822179 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.822521 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.822730 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260810 06:55:43.823953 29459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:43.824548 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.811498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.22.11.190:39773)
0810 06:55:43.811640 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.811645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.811696 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:55:43.811854 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.811859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.811920 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.811993 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.813107 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.813804 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.814964 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.815376 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.815477 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.819646 (+ 4169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.819651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.819654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.819683 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.822115 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.822708 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.822712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.822714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.822832 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.823229 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.823234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.823237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.823551 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.823806 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.824097 (+ 291us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.824279 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:55:43.837291 29467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.824819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36270)
0810 06:55:43.824965 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.824983 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.825075 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.825328 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.825332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.825712 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.825842 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.825849 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.826857 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.826862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.828131 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.828145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.829060 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.829066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.829234 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.829751 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.829773 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.832082 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:43.834921 (+ 2839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.834926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.834930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.835270 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.835636 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.835639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.835642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.835770 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.836315 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:43.836320 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:43.836458 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 06:55:43.836590 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 06:55:43.836837 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 06:55:43.837297 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.824862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.22.11.188:42825)
0810 06:55:43.825036 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.825042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.825059 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:55:43.825163 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.825167 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.825214 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.825282 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.825969 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.826739 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.828254 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.828896 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.829022 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.832241 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.832245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.832249 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.832275 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.834810 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.835370 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.835375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.835377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.835517 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.835873 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.835877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.835880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.836187 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.836322 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.836740 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.836984 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":70}
I20260810 06:55:43.841310 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.826148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.22.11.189:34111)
0810 06:55:43.826292 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.826297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.826366 (+ 69us) server_negotiation.cc:407] Connection header received
0810 06:55:43.826520 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.826526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.826586 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.826660 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.828091 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.829158 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.830777 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.831262 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.831382 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.835511 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.835517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.835521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.835556 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.838674 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.839297 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.839303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.839307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.839443 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.839791 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.839796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.839799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.840148 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.840423 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.840782 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.841020 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 06:55:43.842702 29459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: Bootstrap starting.
I20260810 06:55:43.845981 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:43.846210 29459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:43.847270 29459 log.cc:824] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:43.850466 29459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: No bootstrap required, opened a new log
I20260810 06:55:43.848788 29165 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f3b2cf6f15044dd98c032bede094dedd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33914
I20260810 06:55:43.860997 29459 raft_consensus.cc:348] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:43.861397 29459 raft_consensus.cc:729] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d155a3058f1a4288aaeee95ffb52ebd6, State: Initialized, Role: FOLLOWER
I20260810 06:55:43.861881 29459 consensus_queue.cc:260] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:43.862756 29469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:43.863058 29469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:43.863793 29459 sys_catalog.cc:564] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 06:55:43.872561 29489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:55:43.872705 29489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 06:55:43.876744 29210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 4
I20260810 06:55:43.870707 29341 master_runner.cc:186] Error getting master registration for 127.22.11.190:39773: OK, instance_id { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" instance_seqno: 1786344942745668 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:55:43.878896 29341 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344942, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
I20260810 06:55:43.891294 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.877387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36278 (local address 127.22.11.188:42825)
0810 06:55:43.877555 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.877565 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.877689 (+ 124us) server_negotiation.cc:407] Connection header received
0810 06:55:43.877856 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.877862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.877941 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.878057 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.878963 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.879715 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.881454 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.882133 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.882248 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.886108 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.886114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.886117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.886146 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.888815 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.889409 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.889413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.889415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.889532 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.889904 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.889909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.889912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.890251 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.890377 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.890750 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.890954 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260810 06:55:43.891288 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.877340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36278)
0810 06:55:43.877589 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.877607 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.877735 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.878091 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.878096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.878622 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.878827 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.878840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.879846 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.879852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.881314 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.881330 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.882285 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.882292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.882511 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.883150 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.883174 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.885943 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:43.888928 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.888933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.888938 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.889301 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.889643 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:43.889648 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:43.889652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:43.889806 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:43.890362 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:43.890366 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:43.890518 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 06:55:43.890638 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 06:55:43.890897 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260810 06:55:43.897267 29210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd: Bootstrap starting.
I20260810 06:55:43.901193 29210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:43.902307 29210 log.cc:824] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:43.905498 29210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd: No bootstrap required, opened a new log
I20260810 06:55:43.910807 29210 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:43.911119 29210 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3b2cf6f15044dd98c032bede094dedd, State: Initialized, Role: FOLLOWER
I20260810 06:55:43.911545 29210 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:43.912794 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:43.913383 29210 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:43.913563 29165 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:55:43.913781 29165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:43.915277 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:43.917452 29165 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:43.919266 29165 leader_election.cc:288] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [CANDIDATE]: Term 1 election: Requested vote from peers 8afd572643a342879e1bd394a4a3ad73 (127.22.11.189:34111), d155a3058f1a4288aaeee95ffb52ebd6 (127.22.11.188:42825)
I20260810 06:55:43.921160 29413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3b2cf6f15044dd98c032bede094dedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d155a3058f1a4288aaeee95ffb52ebd6"
I20260810 06:55:43.921674 29413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:43.925110 29413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3b2cf6f15044dd98c032bede094dedd in term 1.
I20260810 06:55:43.927220 29123 leader_election.cc:302] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d155a3058f1a4288aaeee95ffb52ebd6, f3b2cf6f15044dd98c032bede094dedd; no voters:
I20260810 06:55:43.932202 29492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:43.939877 29492 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [term 1 LEADER]: Becoming Leader. State: Replica: f3b2cf6f15044dd98c032bede094dedd, State: Running, Role: LEADER
I20260810 06:55:43.940474 29492 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:43.947625 29502 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3b2cf6f15044dd98c032bede094dedd. Latest consensus state: current_term: 1 leader_uuid: "f3b2cf6f15044dd98c032bede094dedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:43.948069 29502 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:43.974045 29507 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 06:55:43.982049 29330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } attempt: 5
W20260810 06:55:43.986322 29127 tablet.cc:2369] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:55:43.989092 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.949517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36282 (local address 127.22.11.188:42825)
0810 06:55:43.949703 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.949709 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.960586 (+ 10877us) server_negotiation.cc:407] Connection header received
0810 06:55:43.960819 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.960825 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.960923 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.961011 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.970728 (+ 9717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.971780 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.975332 (+ 3552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.975976 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.976100 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.980165 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.980172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.980177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.980219 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.984077 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.984899 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.984906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.984910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.985088 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.985530 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.985535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.985538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.985907 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.987423 (+ 1516us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.988413 (+ 990us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.988706 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 06:55:43.990969 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.949989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.22.11.190:39773)
0810 06:55:43.950168 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.950174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.957177 (+ 7003us) server_negotiation.cc:407] Connection header received
0810 06:55:43.957464 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.957472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.957559 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.957633 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.970665 (+ 13032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.971802 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.975345 (+ 3543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.976138 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.976233 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.979989 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.979996 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.980001 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.980057 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:43.987096 (+ 7039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.988214 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.988219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.988223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.988408 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:43.988849 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:43.988854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:43.988857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:43.989201 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:43.989466 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:43.990371 (+ 905us) server_negotiation.cc:299] Negotiation successful
0810 06:55:43.990625 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:55:43.992632 29507 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 10 06:55:43 dist-test-slave-f7th krb5kdc[29069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344943, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
I20260810 06:55:44.005486 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.950508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32968 (local address 127.22.11.189:34111)
0810 06:55:43.950684 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.950690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.970849 (+ 20159us) server_negotiation.cc:407] Connection header received
0810 06:55:43.971011 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.971016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.971113 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.971210 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.973429 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.974505 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.978309 (+ 3804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.978956 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.979056 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.997034 (+ 17978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.997040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.997052 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.997095 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:44.001291 (+ 4196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:44.002268 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:44.002274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:44.002278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:44.002440 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:44.002983 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:44.002989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:44.002993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:44.003434 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:44.003700 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.004852 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.005106 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 06:55:44.010939 29372 tablet.cc:2369] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:55:44.018867 29332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.982622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:42825 (local address 127.0.0.1:36288)
0810 06:55:43.982964 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:43.982982 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:43.983082 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:43.989899 (+ 6817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:43.989905 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:43.990451 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:43.990661 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.990675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.992407 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.992413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.993825 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:43.993842 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.994886 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.994893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:43.995143 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.995752 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:43.995776 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:43.999213 (+ 3437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:44.014294 (+ 15081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:44.014300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:44.014305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:44.014897 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:44.016827 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:44.016833 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:44.016837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:44.017006 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:44.017825 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:44.017831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:44.018001 (+ 170us) client_negotiation.cc:769] Sending connection context
0810 06:55:44.018143 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 06:55:44.018449 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260810 06:55:44.021647 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:43.983290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36288 (local address 127.22.11.188:42825)
0810 06:55:43.989574 (+ 6284us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:43.989580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:43.989598 (+ 18us) server_negotiation.cc:407] Connection header received
0810 06:55:43.989648 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:43.989653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:43.989722 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:43.990814 (+ 1092us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:43.990965 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.992240 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.994086 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:43.994707 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:43.999397 (+ 4690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:43.999610 (+ 213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:43.999615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:43.999619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:43.999649 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:44.014079 (+ 14430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:44.015130 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:44.015135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:44.015139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:44.015300 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:44.017174 (+ 1874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:44.017180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:44.017184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:44.017559 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:44.019986 (+ 2427us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.020130 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.021181 (+ 1051us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6178,"thread_start_us":20780,"threads_started":1}
I20260810 06:55:44.022297 29413 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [term 1 FOLLOWER]: Refusing update from remote peer f3b2cf6f15044dd98c032bede094dedd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:55:44.031577 29330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73: Bootstrap starting.
I20260810 06:55:44.035580 29330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:44.027323 29091 master_runner.cc:186] Error getting master registration for 127.22.11.189:34111: OK, instance_id { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" instance_seqno: 1786344943261171 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:55:44.037182 29091 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:55:44.037068 29502 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:55:44.046198 29469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3b2cf6f15044dd98c032bede094dedd. Latest consensus state: current_term: 1 leader_uuid: "f3b2cf6f15044dd98c032bede094dedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:44.046497 29469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:44.048959 29330 log.cc:824] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:44.049401 29527 mvcc.cc:205] Tried to move back new op lower bound from 7316868890687094784 to 7316868890385637376. Current Snapshot: MvccSnapshot[applied={T|T < 7316868890687094784}]
I20260810 06:55:44.057700 29330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73: No bootstrap required, opened a new log
I20260810 06:55:44.064706 29502 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3b2cf6f15044dd98c032bede094dedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:44.065359 29502 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:44.067772 29469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3b2cf6f15044dd98c032bede094dedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:44.068013 29469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:44.069439 29507 catalog_manager.cc:1347] Generated new cluster ID: a04c8bbdb28943b6b42b2f531bd9964c
I20260810 06:55:44.069561 29507 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:44.081755 29507 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:44.083360 29507 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:44.083956 29330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:44.084338 29330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afd572643a342879e1bd394a4a3ad73, State: Initialized, Role: FOLLOWER
I20260810 06:55:44.089339 29330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } }
I20260810 06:55:44.097394 29507 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd: Generated new TSK 0
I20260810 06:55:44.098357 29507 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:44.101150 29330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:44.102569 29534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:44.103027 29534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:44.103536 29292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f3b2cf6f15044dd98c032bede094dedd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8afd572643a342879e1bd394a4a3ad73"
I20260810 06:55:44.104192 29291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:44.107836 29291 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [term 1 FOLLOWER]: Refusing update from remote peer f3b2cf6f15044dd98c032bede094dedd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:55:44.109285 29492 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:55:44.119793 29292 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3b2cf6f15044dd98c032bede094dedd in term 1.
I20260810 06:55:44.173337 29534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3b2cf6f15044dd98c032bede094dedd. Latest consensus state: current_term: 1 leader_uuid: "f3b2cf6f15044dd98c032bede094dedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:44.173712 29534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:44.174124 29545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3b2cf6f15044dd98c032bede094dedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:44.174587 29545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:44.194092 29557 catalog_manager.cc:1259] Loaded cluster ID: a04c8bbdb28943b6b42b2f531bd9964c
I20260810 06:55:44.194264 29557 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73: loading cluster ID for follower catalog manager: success
I20260810 06:55:44.197603 29557 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73: acquiring CA information for follower catalog manager: success
I20260810 06:55:44.204329 29557 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8afd572643a342879e1bd394a4a3ad73: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:55:44.232954 22574 mini_kdc.cc:348] Time spent kdestroy: real 0.127s user 0.002s sys 0.030s
W20260810 06:55:44.282653 29213 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.270092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.22.11.190:39773)
0810 06:55:44.270303 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.270308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.279425 (+ 9117us) server_negotiation.cc:407] Connection header received
0810 06:55:44.280727 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.280734 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.280815 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.280891 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.282379 (+ 1488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33934: BlockingRecv error: recv got EOF from 127.0.0.1:33934 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 06:55:44.284420 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.269765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.22.11.188:42825)
0810 06:55:44.269957 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.269963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.279954 (+ 9991us) server_negotiation.cc:407] Connection header received
0810 06:55:44.281085 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.281091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.281185 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.281261 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.284102 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36292: BlockingRecv error: recv got EOF from 127.0.0.1:36292 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 06:55:44.286916 29338 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.273377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.22.11.189:34111)
0810 06:55:44.273559 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.273565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.285101 (+ 11536us) server_negotiation.cc:407] Connection header received
0810 06:55:44.285663 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.285669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.285768 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.285840 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.286687 (+ 847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32984: BlockingRecv error: recv got EOF from 127.0.0.1:32984 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:55:44.344748 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.328955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33000 (local address 127.22.11.189:34111)
0810 06:55:44.329181 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.329187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.336647 (+ 7460us) server_negotiation.cc:407] Connection header received
0810 06:55:44.337198 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.337205 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.337407 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.337556 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.338312 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.339470 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.341585 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.342269 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.342368 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.343150 (+ 782us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.343861 (+ 711us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.344000 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.344245 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.344346 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 06:55:44.390785 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.381099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.22.11.188:42825)
0810 06:55:44.381310 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.381316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.388676 (+ 7360us) server_negotiation.cc:407] Connection header received
0810 06:55:44.389210 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.389217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.389341 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.389427 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.390531 (+ 1104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36300: BlockingRecv error: recv got EOF from 127.0.0.1:36300 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 06:55:44.436251 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.422497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.22.11.189:34111)
0810 06:55:44.422719 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.422725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.429554 (+ 6829us) server_negotiation.cc:407] Connection header received
0810 06:55:44.430081 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.430088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.430203 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.430272 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.430939 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.432058 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.433961 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.434665 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.434765 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.435379 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.435538 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.435606 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.435837 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.435924 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 06:55:44.471560 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.469305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36310 (local address 127.22.11.188:42825)
0810 06:55:44.469499 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.469506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.470043 (+ 537us) server_negotiation.cc:407] Connection header received
0810 06:55:44.470269 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.470275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.470366 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.470466 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.471337 (+ 871us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36310: BlockingRecv error: recv got EOF from 127.0.0.1:36310 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 06:55:44.491058 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.482715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.22.11.189:34111)
0810 06:55:44.482909 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.482915 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.485097 (+ 2182us) server_negotiation.cc:407] Connection header received
0810 06:55:44.485253 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.485259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.485346 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.485446 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.485939 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.487145 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.489045 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.489692 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.489827 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.490251 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.490404 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.490493 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.490628 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.490717 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260810 06:55:44.502774 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.497663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.22.11.188:42825)
0810 06:55:44.497826 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.497832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.501255 (+ 3423us) server_negotiation.cc:407] Connection header received
0810 06:55:44.501461 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.501468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.501566 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.501674 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.502530 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36312: BlockingRecv error: recv got EOF from 127.0.0.1:36312 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:55:44.521327 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.514302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.22.11.189:34111)
0810 06:55:44.514523 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.514530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.515076 (+ 546us) server_negotiation.cc:407] Connection header received
0810 06:55:44.515253 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.515258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.515353 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.515457 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.515954 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.517202 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.519070 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.519813 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.519982 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.520420 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.520561 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.520697 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.520920 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.521010 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
W20260810 06:55:44.530550 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.528065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.22.11.188:42825)
0810 06:55:44.528309 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.528315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.529148 (+ 833us) server_negotiation.cc:407] Connection header received
0810 06:55:44.529306 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.529311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.529405 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.529511 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.530330 (+ 819us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36320: BlockingRecv error: recv got EOF from 127.0.0.1:36320 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
I20260810 06:55:44.548537 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.542786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33020 (local address 127.22.11.189:34111)
0810 06:55:44.542982 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.542988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.543514 (+ 526us) server_negotiation.cc:407] Connection header received
0810 06:55:44.543699 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.543705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.543785 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.543885 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.544414 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.545370 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.546777 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.547255 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.547367 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.547765 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.547878 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.547969 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.548180 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.548261 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 06:55:44.560309 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.558217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.22.11.188:42825)
0810 06:55:44.558413 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.558419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.558902 (+ 483us) server_negotiation.cc:407] Connection header received
0810 06:55:44.559147 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.559153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.559225 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.559305 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.560047 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36330: BlockingRecv error: recv got EOF from 127.0.0.1:36330 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:55:44.584256 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.579097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.22.11.189:34111)
0810 06:55:44.579287 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.579294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.579781 (+ 487us) server_negotiation.cc:407] Connection header received
0810 06:55:44.579964 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.579968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.580037 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.580113 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.580520 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.581340 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.582597 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.583076 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.583180 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.583594 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.583708 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.583781 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.583922 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.583998 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:55:44.586205 29492 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3b2cf6f15044dd98c032bede094dedd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3b2cf6f15044dd98c032bede094dedd" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 39773 } } peers { permanent_uuid: "8afd572643a342879e1bd394a4a3ad73" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 34111 } } peers { permanent_uuid: "d155a3058f1a4288aaeee95ffb52ebd6" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 42825 } } }
I20260810 06:55:44.586529 29492 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3b2cf6f15044dd98c032bede094dedd [sys.catalog]: This master's current role is: LEADER
W20260810 06:55:44.593145 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.591128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.22.11.188:42825)
0810 06:55:44.591290 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.591297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.591955 (+ 658us) server_negotiation.cc:407] Connection header received
0810 06:55:44.592117 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.592123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.592196 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.592276 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.592969 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36344: BlockingRecv error: recv got EOF from 127.0.0.1:36344 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 06:55:44.635473 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.630146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.22.11.189:34111)
0810 06:55:44.630381 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.630388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.630890 (+ 502us) server_negotiation.cc:407] Connection header received
0810 06:55:44.631111 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.631117 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.631209 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.631288 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.631708 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.632457 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.633778 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.634253 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.634363 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.634741 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.634851 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.634926 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.635079 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.635163 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 06:55:44.643165 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.641201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.22.11.188:42825)
0810 06:55:44.641402 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.641410 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.641875 (+ 465us) server_negotiation.cc:407] Connection header received
0810 06:55:44.642052 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.642058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.642127 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.642208 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.642946 (+ 738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36346: BlockingRecv error: recv got EOF from 127.0.0.1:36346 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:55:44.715298 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.710757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.22.11.189:34111)
0810 06:55:44.710928 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.710932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.711288 (+ 356us) server_negotiation.cc:407] Connection header received
0810 06:55:44.711454 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.711457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.711526 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.711608 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.711929 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.712641 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.713912 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.714303 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.714397 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.714749 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.714854 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.714914 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.715013 (+ 99us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.715081 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 06:55:44.722136 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.720424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36352 (local address 127.22.11.188:42825)
0810 06:55:44.720682 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.720688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.720913 (+ 225us) server_negotiation.cc:407] Connection header received
0810 06:55:44.721091 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.721095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.721165 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.721245 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.721966 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36352: BlockingRecv error: recv got EOF from 127.0.0.1:36352 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:55:44.858446 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:44.853597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.22.11.189:34111)
0810 06:55:44.853816 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.853822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.854123 (+ 301us) server_negotiation.cc:407] Connection header received
0810 06:55:44.854313 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.854317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.854386 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.854462 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:44.854896 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.855598 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.856909 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:44.857298 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:44.857409 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:44.857775 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:44.857887 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:44.857975 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:44.858111 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 06:55:44.858191 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 06:55:44.865516 29466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:44.863580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36368 (local address 127.22.11.188:42825)
0810 06:55:44.863824 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:44.863829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:44.864077 (+ 248us) server_negotiation.cc:407] Connection header received
0810 06:55:44.864254 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:44.864260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:44.864335 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:44.864426 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:44.865304 (+ 878us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36368: BlockingRecv error: recv got EOF from 127.0.0.1:36368 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 06:55:44.874500 29489 catalog_manager.cc:1259] Loaded cluster ID: a04c8bbdb28943b6b42b2f531bd9964c
I20260810 06:55:44.874581 29489 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: loading cluster ID for follower catalog manager: success
I20260810 06:55:44.877022 29489 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: acquiring CA information for follower catalog manager: success
I20260810 06:55:44.878693 29489 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d155a3058f1a4288aaeee95ffb52ebd6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:55:45.130950 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:45.126039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.22.11.189:34111)
0810 06:55:45.126280 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:45.126287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:45.126473 (+ 186us) server_negotiation.cc:407] Connection header received
0810 06:55:45.126701 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:45.126706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:45.126774 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:45.126870 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:45.127338 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.128061 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.129392 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.129810 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.129922 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:45.130295 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:45.130401 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:45.130475 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:45.130632 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 06:55:45.130711 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 06:55:45.141186 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:45.136240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36384 (local address 127.22.11.188:42825)
0810 06:55:45.136440 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:45.136445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:45.136644 (+ 199us) server_negotiation.cc:407] Connection header received
0810 06:55:45.136815 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:45.136819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:45.136978 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:45.137068 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:45.137393 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.138052 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.139290 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.139674 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.139761 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:45.140138 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:45.140693 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:45.140800 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:45.140838 (+ 38us) server_negotiation.cc:299] Negotiation successful
0810 06:55:45.140900 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 06:55:45.154963 29338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:45.147682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33072 (local address 127.22.11.189:34111)
0810 06:55:45.147882 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:45.147888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:45.148870 (+ 982us) server_negotiation.cc:407] Connection header received
0810 06:55:45.149526 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:45.149530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:45.149619 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:45.149718 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:45.150331 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.151390 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.153291 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.153673 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.153780 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:45.154245 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:45.154353 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:45.154420 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:45.154574 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 06:55:45.154659 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:55:45.155740 29466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:45.147458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.22.11.188:42825)
0810 06:55:45.147661 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:45.147667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:45.149577 (+ 1910us) server_negotiation.cc:407] Connection header received
0810 06:55:45.149691 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:45.149696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:45.149770 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:45.149849 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:45.150510 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.151530 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.153917 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.154418 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.154543 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:45.155026 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:45.155159 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:45.155238 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:45.155446 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 06:55:45.155512 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 06:55:45.157141 29740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:45.147157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.22.11.190:39773)
0810 06:55:45.148434 (+ 1277us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:45.148440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:45.149703 (+ 1263us) server_negotiation.cc:407] Connection header received
0810 06:55:45.149838 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:45.149844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:45.150064 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:45.150251 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:45.150757 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.151429 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.154639 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:45.155233 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:45.155350 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:45.155924 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:45.156399 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:45.156483 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:45.156825 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 06:55:45.156907 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":1031,"threads_started":1}
I20260810 06:55:45.161197 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29091
I20260810 06:55:45.162755 29203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:45.273171 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29091
W20260810 06:55:45.297045 29367 connection.cc:535] server connection from 127.0.0.1:36278 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:55:45.297454 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29218
I20260810 06:55:45.298476 29322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:45.404778 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29218
I20260810 06:55:45.430873 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29341
I20260810 06:55:45.431967 29453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:45.538138 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29341
2026-08-10T06:55:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4781 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 06:55:45 dist-test-slave-f7th krb5kdc[29750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 06:55:45 dist-test-slave-f7th krb5kdc[29750](info): set up 2 sockets
Aug 10 06:55:45 dist-test-slave-f7th krb5kdc[29750](info): commencing operation
krb5kdc: starting...
W20260810 06:55:46.801100 22574 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/test-user.keytab.
Aug 10 06:55:46 dist-test-slave-f7th krb5kdc[29750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T06:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T06:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.22.11.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
Entry for principal kudu/127.22.11.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab.
I20260810 06:55:47.093474 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40563
--webserver_interface=127.22.11.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40563,127.22.11.189:37235,127.22.11.188:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 06:55:47.425854 29776 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:55:47.426154 29776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:47.426225 29776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:47.426304 29776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:47.433251 29776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:47.433336 29776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:47.433390 29776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:47.433439 29776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:47.433499 29776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:47.442917 29776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40563,127.22.11.189:37235,127.22.11.188:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.190.keytab
--principal=kudu/127.22.11.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.190:40563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.22.11.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:47.444072 29776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:47.445379 29776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:47.456301 29789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:47.456486 29785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:47.456308 29786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:47.457345 29776 server_base.cc:1034] running on GCE node
I20260810 06:55:47.458055 29776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:47.459300 29776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:47.460501 29776 hybrid_clock.cc:648] HybridClock initialized: now 1786344947460436 us; error 40 us; skew 500 ppm
I20260810 06:55:47.460817 29776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:47 dist-test-slave-f7th krb5kdc[29750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:47.463759 29776 init.cc:377] Logged in from keytab as kudu/127.22.11.190@KRBTEST.COM (short username kudu)
I20260810 06:55:47.465111 29776 webserver.cc:460] Webserver started at http://127.22.11.190:35407/ using document root <none> and password file <none>
I20260810 06:55:47.465728 29776 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:47.465821 29776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:47.466125 29776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:47.468998 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data/instance:
uuid: "6eb06327f7a146c1bc305c2eac1e3a8f"
format_stamp: "Formatted at 2026-08-10 06:55:47 on dist-test-slave-f7th"
server_key: "e4bea97d6bab70803579bab9f320fb82"
server_key_iv: "e515d906207821b85c3617de60461c4d"
server_key_version: "encryptionkey@0"
I20260810 06:55:47.469743 29776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/wal/instance:
uuid: "6eb06327f7a146c1bc305c2eac1e3a8f"
format_stamp: "Formatted at 2026-08-10 06:55:47 on dist-test-slave-f7th"
server_key: "e4bea97d6bab70803579bab9f320fb82"
server_key_iv: "e515d906207821b85c3617de60461c4d"
server_key_version: "encryptionkey@0"
I20260810 06:55:47.474447 29776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 06:55:47.477614 29801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:47.478778 29776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 06:55:47.478928 29776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/wal
uuid: "6eb06327f7a146c1bc305c2eac1e3a8f"
format_stamp: "Formatted at 2026-08-10 06:55:47 on dist-test-slave-f7th"
server_key: "e4bea97d6bab70803579bab9f320fb82"
server_key_iv: "e515d906207821b85c3617de60461c4d"
server_key_version: "encryptionkey@0"
I20260810 06:55:47.479120 29776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:47.495015 29776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:47.498483 29776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:47.498718 29776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:47.515619 29776 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.190:40563
I20260810 06:55:47.515640 29887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.190:40563 every 8 connection(s)
I20260810 06:55:47.516906 29776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/data/info.pb
I20260810 06:55:47.520421 29888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:47.522467 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29776
I20260810 06:55:47.522653 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-0/wal/instance
I20260810 06:55:47.523094 22574 external_mini_cluster.cc:1428] Setting key ce94835741815aaa1f539093d90ad1a8
I20260810 06:55:47.526944 29888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:47 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:47.548203 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.531513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.22.11.190:40563)
0810 06:55:47.532183 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:47.532191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:47.532247 (+ 56us) server_negotiation.cc:407] Connection header received
0810 06:55:47.533350 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:47.533355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:47.533575 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:47.533756 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:47.535000 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:47.536180 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:47.537538 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:47.538074 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:47.538239 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:47.541675 (+ 3436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:47.541683 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:47.541694 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:47.541727 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:47.544244 (+ 2517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:47.545144 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:47.545150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:47.545152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:47.545294 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:47.545689 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:47.545692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:47.545694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:47.546138 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:47.546293 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:47.546895 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 06:55:47.547209 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":485,"thread_start_us":306,"threads_started":1}
I20260810 06:55:47.548237 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.530207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40563 (local address 127.0.0.1:39002)
0810 06:55:47.530966 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.531009 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:47.533007 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:47.533785 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:47.533791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:47.534458 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:47.534792 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:47.534809 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:47.536319 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:47.536323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:47.537403 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:47.537421 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:47.538283 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:47.538290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:47.538485 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:47.539108 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:47.539135 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:47.541454 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:47.544379 (+ 2925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:47.544394 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:47.544407 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:47.545019 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:47.545401 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:47.545406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:47.545409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:47.545598 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:47.546277 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:47.546284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:47.546502 (+ 218us) client_negotiation.cc:769] Sending connection context
0810 06:55:47.546761 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 06:55:47.547372 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":130,"thread_start_us":315,"threads_started":1}
I20260810 06:55:47.551225 29888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:47.552274 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.551926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:59978)
0810 06:55:47.552078 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.552178 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 06:55:47.552554 29805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
I20260810 06:55:47.554025 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.553703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:59990)
0810 06:55:47.553832 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.553896 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 06:55:47.554206 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:37235: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.22.11.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab.
Entry for principal kudu/127.22.11.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab.
I20260810 06:55:47.596879 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:37235
--webserver_interface=127.22.11.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab
--principal=kudu/127.22.11.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40563,127.22.11.189:37235,127.22.11.188:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:55:47.603502 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } attempt: 1
I20260810 06:55:47.604606 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.604141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:59992)
0810 06:55:47.604338 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.604468 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 06:55:47.605633 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.605262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60004)
0810 06:55:47.605406 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.605476 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:55:47.605818 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:37235: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
I20260810 06:55:47.674081 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } attempt: 2
I20260810 06:55:47.675386 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.674811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60012)
0810 06:55:47.675086 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.675205 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260810 06:55:47.676476 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.676069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60020)
0810 06:55:47.676212 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.676318 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 06:55:47.676707 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:37235: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
I20260810 06:55:47.768014 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } attempt: 3
I20260810 06:55:47.769367 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.768839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60032)
0810 06:55:47.769077 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.769192 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 06:55:47.770426 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.770036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60040)
0810 06:55:47.770187 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.770269 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 06:55:47.770617 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:37235: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
I20260810 06:55:47.913866 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } attempt: 4
I20260810 06:55:47.915174 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.914550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60052)
0810 06:55:47.914892 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.915002 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174}
I20260810 06:55:47.916251 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:47.915845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60058)
0810 06:55:47.915996 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:47.916098 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:55:47.916509 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.189:37235: Network error: Client connection negotiation failed: client connection to 127.22.11.189:37235: connect: Connection refused (error 111)
W20260810 06:55:47.936373 29905 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:55:47.936707 29905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:47.936793 29905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:47.936885 29905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:47.943939 29905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:47.944020 29905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 06:55:47.944083 29905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:47.944132 29905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:47.944192 29905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:47.953939 29905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40563,127.22.11.189:37235,127.22.11.188:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.189.keytab
--principal=kudu/127.22.11.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.189:37235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.22.11.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:47.955271 29905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:47.956655 29905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:47.968011 29912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:47.967972 29911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:47.967926 29915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:47.969152 29905 server_base.cc:1034] running on GCE node
I20260810 06:55:47.969887 29905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:47.971202 29905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:47.972450 29905 hybrid_clock.cc:648] HybridClock initialized: now 1786344947972379 us; error 39 us; skew 500 ppm
I20260810 06:55:47.972800 29905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:47 dist-test-slave-f7th krb5kdc[29750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:47.980671 29905 init.cc:377] Logged in from keytab as kudu/127.22.11.189@KRBTEST.COM (short username kudu)
I20260810 06:55:47.982115 29905 webserver.cc:460] Webserver started at http://127.22.11.189:40413/ using document root <none> and password file <none>
I20260810 06:55:47.982787 29905 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:47.982882 29905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:47.983234 29905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:47.986235 29905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data/instance:
uuid: "52dd6888e1524e6e8715d037ab803d1e"
format_stamp: "Formatted at 2026-08-10 06:55:47 on dist-test-slave-f7th"
server_key: "678c9106900c3afb5c8eb42907540c03"
server_key_iv: "e58b6553d12389872cab21c56b1bb2f6"
server_key_version: "encryptionkey@0"
I20260810 06:55:47.987037 29905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/wal/instance:
uuid: "52dd6888e1524e6e8715d037ab803d1e"
format_stamp: "Formatted at 2026-08-10 06:55:47 on dist-test-slave-f7th"
server_key: "678c9106900c3afb5c8eb42907540c03"
server_key_iv: "e58b6553d12389872cab21c56b1bb2f6"
server_key_version: "encryptionkey@0"
I20260810 06:55:47.992244 29905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 06:55:47.995940 29924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:47.997337 29905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:55:47.997509 29905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/wal
uuid: "52dd6888e1524e6e8715d037ab803d1e"
format_stamp: "Formatted at 2026-08-10 06:55:47 on dist-test-slave-f7th"
server_key: "678c9106900c3afb5c8eb42907540c03"
server_key_iv: "e58b6553d12389872cab21c56b1bb2f6"
server_key_version: "encryptionkey@0"
I20260810 06:55:47.997721 29905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:48.016660 29905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:48.020610 29905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:48.020937 29905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:48.038282 29905 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.189:37235
I20260810 06:55:48.038290 30017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.189:37235 every 8 connection(s)
I20260810 06:55:48.039618 29905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/data/info.pb
I20260810 06:55:48.043195 30018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:48.049695 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 29905
I20260810 06:55:48.049863 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-1/wal/instance
I20260810 06:55:48.049464 30018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:48.050307 22574 external_mini_cluster.cc:1428] Setting key 4da6bb2cba2610d176a49e032d7e2629
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:48.068518 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.051960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.22.11.190:40563)
0810 06:55:48.052381 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.052386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.053471 (+ 1085us) server_negotiation.cc:407] Connection header received
0810 06:55:48.055264 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.055270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.055350 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.055448 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.056973 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.057675 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.059139 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.059540 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.059650 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.062884 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.062889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.062893 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.062937 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.065529 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.066253 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.066257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.066259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.066392 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.066787 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.066791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.066793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.067085 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.067230 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.067923 (+ 693us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.068144 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":223,"threads_started":1}
I20260810 06:55:48.069379 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.052565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40563 (local address 127.0.0.1:39008)
0810 06:55:48.053294 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.053353 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.055004 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.055678 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.055686 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.056192 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.056764 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.056784 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.057829 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.057833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.058989 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.059002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.059747 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.059754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.059973 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.060574 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.060595 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.062673 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.065659 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.065667 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.065682 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.066130 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.066494 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.066500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.066504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.066673 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.067216 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.067223 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.067560 (+ 337us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.067786 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.068248 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":293,"threads_started":1}
I20260810 06:55:48.071004 30018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.22.11.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.22.11.188@KRBTEST.COM" created.
I20260810 06:55:48.086089 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.071791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60068)
0810 06:55:48.071941 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.071959 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.072054 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.073324 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.073328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.073709 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.073842 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.073850 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.075639 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.075643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.076477 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.076488 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.077524 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.077529 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.077753 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.078273 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.078290 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.080388 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.083679 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.083683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.083686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.084142 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.084484 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.084487 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.084489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.084654 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.085267 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.085272 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.085393 (+ 121us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.085518 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.085724 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":33}
I20260810 06:55:48.086192 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.072052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.22.11.189:37235)
0810 06:55:48.072784 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.072791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.072821 (+ 30us) server_negotiation.cc:407] Connection header received
0810 06:55:48.072915 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.072922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.073133 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.073280 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.074019 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.075509 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.076635 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.077388 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.077521 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.080594 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.080605 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.080651 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.080682 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.083573 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.084270 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.084277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.084279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.084397 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.084751 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.084754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.084756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.085119 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.085266 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.085674 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.085872 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":541,"thread_start_us":400,"threads_started":1}
I20260810 06:55:48.088553 30018 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:48.089668 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.089203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33758)
0810 06:55:48.089360 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.089488 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 06:55:48.089941 29929 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.091351 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.091028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33768)
0810 06:55:48.091167 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.091246 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 06:55:48.091553 30018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.22.11.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab.
Entry for principal kudu/127.22.11.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab.
I20260810 06:55:48.121392 22574 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
/tmp/dist-test-taskJIbnRf/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:44099
--webserver_interface=127.22.11.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab
--principal=kudu/127.22.11.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.22.11.148:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.22.11.190:40563,127.22.11.189:37235,127.22.11.188:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 06:55:48.140873 30018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 1
I20260810 06:55:48.141963 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.141510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33770)
0810 06:55:48.141718 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.141817 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 06:55:48.143029 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.142707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33786)
0810 06:55:48.142862 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.142929 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 06:55:48.143240 30018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.211462 30018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 2
I20260810 06:55:48.212538 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.212080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33796)
0810 06:55:48.212275 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.212363 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 06:55:48.213533 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.213206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33810)
0810 06:55:48.213349 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.213421 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 06:55:48.213730 30018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.215762 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } attempt: 5
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:55:48.229344 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.216421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60074 (local address 127.22.11.189:37235)
0810 06:55:48.216581 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.216587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.216631 (+ 44us) server_negotiation.cc:407] Connection header received
0810 06:55:48.216797 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.216803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.216900 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.217021 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.217740 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.218822 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.219919 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.220600 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.220764 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.223869 (+ 3105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.223873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.223877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.223908 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.226780 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.227599 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.227603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.227605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.227718 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.228122 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.228125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.228127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.228389 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.228518 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.228974 (+ 456us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.229134 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 06:55:48.229633 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.216341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60074)
0810 06:55:48.216518 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.216533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.216664 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.217045 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.217049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.217403 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.217594 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.217604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.218936 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.218940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.219777 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.219790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.220759 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.220764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.220938 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.221490 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.221509 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.223681 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.226911 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.226916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.226920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.227492 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.227817 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.227822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.227826 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.228027 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.228517 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.228524 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.228725 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.228891 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.229208 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 06:55:48.230701 29888 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:48.231819 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.231392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33826)
0810 06:55:48.231568 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.231663 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 06:55:48.232751 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.232389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33830)
0810 06:55:48.232517 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.232633 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260810 06:55:48.232966 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.284230 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 1
I20260810 06:55:48.285508 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.284941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33840)
0810 06:55:48.285181 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.285300 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 06:55:48.286511 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.286165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33842)
0810 06:55:48.286300 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.286382 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260810 06:55:48.286731 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.304920 30018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 3
I20260810 06:55:48.305903 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.305467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33846)
0810 06:55:48.305663 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.305755 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260810 06:55:48.306993 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.306588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33860)
0810 06:55:48.306801 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.306885 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260810 06:55:48.307195 30018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.354970 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 2
I20260810 06:55:48.356208 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.355651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33872)
0810 06:55:48.355889 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.356010 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 06:55:48.357177 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.356827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33886)
0810 06:55:48.356961 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.357056 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260810 06:55:48.357384 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.450377 30018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 4
I20260810 06:55:48.451481 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.451016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33902)
0810 06:55:48.451216 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.451305 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 06:55:48.452484 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.452145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33916)
0810 06:55:48.452282 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.452363 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260810 06:55:48.452733 30018 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
W20260810 06:55:48.456022 30038 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 06:55:48.456331 30038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 06:55:48.456423 30038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 06:55:48.456535 30038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 06:55:48.463553 30038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 06:55:48.463654 30038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260810 06:55:48.463584 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 3
W20260810 06:55:48.463743 30038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 06:55:48.463812 30038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 06:55:48.463896 30038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 06:55:48.464589 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.464138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33926)
0810 06:55:48.464317 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.464413 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 06:55:48.465559 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.465176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33930)
0810 06:55:48.465357 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.465442 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260810 06:55:48.465780 29888 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.22.11.188:44099: Network error: Client connection negotiation failed: client connection to 127.22.11.188:44099: connect: Connection refused (error 111)
I20260810 06:55:48.473956 30038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.22.11.148:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.22.11.190:40563,127.22.11.189:37235,127.22.11.188:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/krb5kdc/kudu_127.22.11.188.keytab
--principal=kudu/127.22.11.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.22.11.188:44099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.22.11.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 05:49:26 UTC on ccycloud.quasar-maxtsy.root.comops.site
build id 4577111
ASAN enabled
I20260810 06:55:48.475081 30038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 06:55:48.476236 30038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 06:55:48.486974 30049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:48.486898 30053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 06:55:48.486936 30050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 06:55:48.487780 30038 server_base.cc:1034] running on GCE node
I20260810 06:55:48.488480 30038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 06:55:48.489806 30038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 06:55:48.491017 30038 hybrid_clock.cc:648] HybridClock initialized: now 1786344948490948 us; error 47 us; skew 500 ppm
I20260810 06:55:48.491317 30038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344948, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 06:55:48.494274 30038 init.cc:377] Logged in from keytab as kudu/127.22.11.188@KRBTEST.COM (short username kudu)
I20260810 06:55:48.495582 30038 webserver.cc:460] Webserver started at http://127.22.11.188:35543/ using document root <none> and password file <none>
I20260810 06:55:48.496187 30038 fs_manager.cc:359] Metadata directory not provided
I20260810 06:55:48.496279 30038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 06:55:48.496582 30038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 06:55:48.499346 30038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data/instance:
uuid: "47ee2f59b265453f91581709d77642eb"
format_stamp: "Formatted at 2026-08-10 06:55:48 on dist-test-slave-f7th"
server_key: "b07da693cd0a3304dd836968f61a90cb"
server_key_iv: "bdd83017d1808e203df8cd2e6e777813"
server_key_version: "encryptionkey@0"
I20260810 06:55:48.500083 30038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/wal/instance:
uuid: "47ee2f59b265453f91581709d77642eb"
format_stamp: "Formatted at 2026-08-10 06:55:48 on dist-test-slave-f7th"
server_key: "b07da693cd0a3304dd836968f61a90cb"
server_key_iv: "bdd83017d1808e203df8cd2e6e777813"
server_key_version: "encryptionkey@0"
I20260810 06:55:48.504885 30038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.007s
I20260810 06:55:48.507997 30061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 06:55:48.509256 30038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 06:55:48.509418 30038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data,/tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/wal
uuid: "47ee2f59b265453f91581709d77642eb"
format_stamp: "Formatted at 2026-08-10 06:55:48 on dist-test-slave-f7th"
server_key: "b07da693cd0a3304dd836968f61a90cb"
server_key_iv: "bdd83017d1808e203df8cd2e6e777813"
server_key_version: "encryptionkey@0"
I20260810 06:55:48.509606 30038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 06:55:48.530251 30038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 06:55:48.534215 30038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 06:55:48.534444 30038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 06:55:48.550889 30038 rpc_server.cc:307] RPC server started. Bound to: 127.22.11.188:44099
I20260810 06:55:48.550933 30152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.22.11.188:44099 every 8 connection(s)
I20260810 06:55:48.552100 30038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/data/info.pb
I20260810 06:55:48.557525 30155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 06:55:48.561226 22574 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu as pid 30038
I20260810 06:55:48.561385 22574 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskJIbnRf/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786344580063516-22574-0/minicluster-data/master-2/wal/instance
I20260810 06:55:48.561777 22574 external_mini_cluster.cc:1428] Setting key 9a578cb9e720192ef7a94342dc30bae1
I20260810 06:55:48.564486 30155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344948, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:48.582521 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.568481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39018 (local address 127.22.11.190:40563)
0810 06:55:48.568712 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.568718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.569254 (+ 536us) server_negotiation.cc:407] Connection header received
0810 06:55:48.569418 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.569422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.569496 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.569576 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.570823 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.571783 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.573528 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.574151 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.574320 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.577435 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.577440 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.577443 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.577475 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.579828 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.580439 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.580443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.580445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.580573 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.580965 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.580969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.580971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.581248 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.581364 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.581953 (+ 589us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.582166 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 06:55:48.583597 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.563346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.22.11.188:44099)
0810 06:55:48.564111 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.564125 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.564176 (+ 51us) server_negotiation.cc:407] Connection header received
0810 06:55:48.565833 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.565861 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.566359 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.566772 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.568165 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.570160 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.571655 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.572533 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.572730 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.577044 (+ 4314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.577059 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.577072 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.577109 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.579734 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.580377 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.580385 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.580388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.580533 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.580956 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.580959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.580961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.581448 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.581752 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.581938 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.582225 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":335,"threads_started":1}
I20260810 06:55:48.583567 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.568390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.190:40563 (local address 127.0.0.1:39018)
0810 06:55:48.569120 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.569160 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.569314 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.569654 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.569660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.570427 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.570675 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.570684 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.571918 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.571924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.573368 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.573383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.574359 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.574364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.574477 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.574996 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.575016 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.577239 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.579946 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.579954 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.579961 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.580320 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.580693 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.580698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.580702 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.580848 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.581361 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.581368 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.581564 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.581821 (+ 257us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.582433 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":159,"thread_start_us":319,"threads_started":1}
I20260810 06:55:48.585167 30155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344948, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.190@KRBTEST.COM
I20260810 06:55:48.597168 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.586300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.189:37235 (local address 127.0.0.1:60090)
0810 06:55:48.586507 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.586520 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.586610 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.586886 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.586891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.587222 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.587352 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.587360 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.588275 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.588280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.589167 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.589179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.589824 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.589830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.590009 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.590443 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.590458 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.592270 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.595140 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.595145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.595149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.595471 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.595807 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.595812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.595816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.595940 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.596393 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.596398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.596545 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.596685 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.596868 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260810 06:55:48.597540 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.585928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.22.11.189:37235)
0810 06:55:48.586185 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.586192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.586590 (+ 398us) server_negotiation.cc:407] Connection header received
0810 06:55:48.586704 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.586709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.586769 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.586855 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.587481 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.588151 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.589290 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.589701 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.589803 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.592431 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.592435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.592438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.592467 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.595023 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.595581 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.595585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.595587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.595707 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.596036 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.596041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.596044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.596284 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.596412 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.596796 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.597032 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 06:55:48.598414 30155 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } has no permanent_uuid. Determining permanent_uuid...
I20260810 06:55:48.598989 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.586342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39022 (local address 127.22.11.190:40563)
0810 06:55:48.586543 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.586548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.586565 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:55:48.586650 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.586655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.586716 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.586790 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.587875 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.588537 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.589697 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.590127 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.590233 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.593881 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.593886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.593889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.593918 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.596309 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.597005 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.597010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.597014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.597153 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.597568 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.597571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.597573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.597868 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.598092 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.598507 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.598704 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344948, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.188@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.22.11.189@KRBTEST.COM
I20260810 06:55:48.611756 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.599070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.22.11.188:44099)
0810 06:55:48.599208 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.599213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.599334 (+ 121us) server_negotiation.cc:407] Connection header received
0810 06:55:48.599482 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.599487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.599536 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.599606 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.600333 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.601275 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.602753 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.603345 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.603437 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.606428 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.606434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.606439 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.606473 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.609276 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.609977 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.609982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.609986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.610113 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.610431 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.610437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.610440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.610736 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.610840 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.611242 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.611476 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":52}
I20260810 06:55:48.612017 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.599054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33952)
0810 06:55:48.599260 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.599275 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.599380 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.599635 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.599639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.600032 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.600180 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.600192 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.601385 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.601391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.602603 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.602615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.603499 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.603505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.603725 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.604196 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.604217 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.606242 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.609406 (+ 3164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.609411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.609414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.609872 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.610212 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.610215 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.610217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.610332 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.610856 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.610862 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.611015 (+ 153us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.611176 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.611647 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":142}
I20260810 06:55:48.615357 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.600410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60098 (local address 127.22.11.189:37235)
0810 06:55:48.600565 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.600570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.600587 (+ 17us) server_negotiation.cc:407] Connection header received
0810 06:55:48.600739 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.600744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.600806 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.600890 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.602277 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.603236 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.604882 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.605264 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.605370 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.609883 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.609887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.609891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.609919 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.613054 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.613728 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.613732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.613734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.613857 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.614270 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.614274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.614275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.614505 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.614688 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.614978 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.615140 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 06:55:48.617429 30155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: Bootstrap starting.
I20260810 06:55:48.619505 22574 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 06:55:48.620543 30155 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:48.621567 30155 log.cc:824] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:48.624521 30155 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: No bootstrap required, opened a new log
I20260810 06:55:48.621770 29850 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39022
I20260810 06:55:48.634908 30155 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.635293 30155 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47ee2f59b265453f91581709d77642eb, State: Initialized, Role: FOLLOWER
I20260810 06:55:48.635766 30155 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.636680 30164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.636996 30164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:48.637756 30155 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:48.642961 29888 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 4
W20260810 06:55:48.647507 30180 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 06:55:48.647607 30180 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.190@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
I20260810 06:55:48.645422 30038 master_runner.cc:186] Error getting master registration for 127.22.11.190:40563: OK, instance_id { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" instance_seqno: 1786344947510697 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:55:48.658093 30038 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:55:48.660213 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.643566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33958)
0810 06:55:48.643931 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.643952 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.644100 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.644558 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.644563 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.645168 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.645368 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.645380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.646907 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.646913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.648311 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.648327 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.649150 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.649155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.649358 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.650105 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.650129 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.653274 (+ 3145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.657227 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.657233 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.657237 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.657730 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.658221 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.658226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.658229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.658400 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.659186 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.659191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.659370 (+ 179us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.659522 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.659822 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237}
I20260810 06:55:48.660246 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.643734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33958 (local address 127.22.11.188:44099)
0810 06:55:48.643932 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.643938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.644258 (+ 320us) server_negotiation.cc:407] Connection header received
0810 06:55:48.644320 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.644324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.644396 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.644504 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.645534 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.646739 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.648449 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.649020 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.649129 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.653478 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.653484 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.653489 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.653527 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.657070 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.657911 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.657916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.657919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.658079 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.658523 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.658528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.658548 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.658952 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.659116 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.659665 (+ 549us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.659891 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 06:55:48.666208 29888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f: Bootstrap starting.
I20260810 06:55:48.670267 29888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:48.671558 29888 log.cc:824] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:48.674793 29888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f: No bootstrap required, opened a new log
I20260810 06:55:48.682034 29888 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.682423 29888 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb06327f7a146c1bc305c2eac1e3a8f, State: Initialized, Role: FOLLOWER
I20260810 06:55:48.682924 29888 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.683844 30184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.684243 30184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:48.685150 29888 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:48.685218 29850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 06:55:48.685400 29850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:48.688017 29850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.690507 30106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "47ee2f59b265453f91581709d77642eb"
I20260810 06:55:48.691004 30106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:48.694346 30106 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb06327f7a146c1bc305c2eac1e3a8f in term 1.
I20260810 06:55:48.694725 29850 leader_election.cc:288] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [CANDIDATE]: Term 1 election: Requested vote from peers 52dd6888e1524e6e8715d037ab803d1e (127.22.11.189:37235), 47ee2f59b265453f91581709d77642eb (127.22.11.188:44099)
I20260810 06:55:48.694998 29802 leader_election.cc:302] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47ee2f59b265453f91581709d77642eb, 6eb06327f7a146c1bc305c2eac1e3a8f; no voters:
I20260810 06:55:48.695436 30184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 06:55:48.708606 30184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [term 1 LEADER]: Becoming Leader. State: Replica: 6eb06327f7a146c1bc305c2eac1e3a8f, State: Running, Role: LEADER
I20260810 06:55:48.709820 30184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.724434 30196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb06327f7a146c1bc305c2eac1e3a8f. Latest consensus state: current_term: 1 leader_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.725055 30196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:48.734956 30217 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 06:55:48.740027 30214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260810 06:55:48.751281 29815 tablet.cc:2369] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:55:48.751894 30018 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } attempt: 5
I20260810 06:55:48.739202 29776 master_runner.cc:186] Error getting master registration for 127.22.11.189:37235: OK, instance_id { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" instance_seqno: 1786344948033436 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 06:55:48.753450 29776 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 06:55:48.761025 30214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 06:55:48.767102 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.726730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39032 (local address 127.22.11.190:40563)
0810 06:55:48.726953 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.726960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.738585 (+ 11625us) server_negotiation.cc:407] Connection header received
0810 06:55:48.738862 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.738867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.738955 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.739034 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.744173 (+ 5139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.745315 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.747223 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.748004 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.748105 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.753069 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.753075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.753079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.753116 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.756967 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.764720 (+ 7753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.764726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.764729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.764894 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.765364 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.765369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.765372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.765712 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.765955 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.766517 (+ 562us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.766732 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 10 06:55:48 dist-test-slave-f7th krb5kdc[29750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786344947, etypes {rep=17 tkt=17 ses=17}, kudu/127.22.11.189@KRBTEST.COM for kudu/127.22.11.188@KRBTEST.COM
I20260810 06:55:48.783618 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.752417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.22.11.188:44099 (local address 127.0.0.1:33984)
0810 06:55:48.752782 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0810 06:55:48.752801 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 06:55:48.752897 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 06:55:48.763010 (+ 10113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 06:55:48.763016 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 06:55:48.763518 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0810 06:55:48.763727 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.763739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.766689 (+ 2950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.766694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.768110 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 06:55:48.768126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.769891 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.769898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 06:55:48.770113 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.770664 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 06:55:48.770687 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 06:55:48.773454 (+ 2767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 06:55:48.780110 (+ 6656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.780114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.780118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.780667 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.781382 (+ 715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 06:55:48.781387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 06:55:48.781391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 06:55:48.781560 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 06:55:48.782622 (+ 1062us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 06:55:48.782627 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 06:55:48.782783 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 06:55:48.782913 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 06:55:48.783215 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260810 06:55:48.784189 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.728347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.22.11.189:37235)
0810 06:55:48.728541 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.728547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.745005 (+ 16458us) server_negotiation.cc:407] Connection header received
0810 06:55:48.745397 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.745402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.745481 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.745560 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.747646 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.748672 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.764720 (+ 16048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.765365 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.765453 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.775564 (+ 10111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.775571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.775575 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.775611 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.779055 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.780599 (+ 1544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.780604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.780607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.780787 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.782249 (+ 1462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.782254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.782257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.782639 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.782863 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.783551 (+ 688us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.783888 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 06:55:48.785696 30222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.753086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33984 (local address 127.22.11.188:44099)
0810 06:55:48.762651 (+ 9565us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.762657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.762684 (+ 27us) server_negotiation.cc:407] Connection header received
0810 06:55:48.762742 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.762747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.762832 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.762950 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.764270 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.766437 (+ 2167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.768505 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.769670 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.770442 (+ 772us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.773926 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.773935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.773940 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.774003 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.779914 (+ 5911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.780876 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.780881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.780884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.781200 (+ 316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.781743 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.781748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.781752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.782355 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.784213 (+ 1858us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.784604 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.784986 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9404,"thread_start_us":10084,"threads_started":1}
W20260810 06:55:48.786314 30071 tablet.cc:2369] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 06:55:48.789852 30106 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [term 1 FOLLOWER]: Refusing update from remote peer 6eb06327f7a146c1bc305c2eac1e3a8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:55:48.790925 30196 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:55:48.798570 30018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e: Bootstrap starting.
I20260810 06:55:48.803952 30018 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e: Neither blocks nor log segments found. Creating new log.
I20260810 06:55:48.808250 30164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb06327f7a146c1bc305c2eac1e3a8f. Latest consensus state: current_term: 1 leader_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.808595 30164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:48.821013 30018 log.cc:824] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e: Log is configured to *not* fsync() on all Append() calls
I20260810 06:55:48.821171 30164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.821411 30164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:48.827185 30018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e: No bootstrap required, opened a new log
I20260810 06:55:48.829038 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:48.738347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.22.11.188:44099)
0810 06:55:48.738542 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:48.738548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:48.751004 (+ 12456us) server_negotiation.cc:407] Connection header received
0810 06:55:48.763521 (+ 12517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:48.763527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:48.763599 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:48.763665 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:48.770659 (+ 6994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.771736 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.774505 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:48.775210 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:48.775301 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:48.816206 (+ 40905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 06:55:48.816214 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 06:55:48.816219 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 06:55:48.816265 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 06:55:48.820160 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.823051 (+ 2891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.823057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.823061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.823212 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 06:55:48.824285 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 06:55:48.824290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 06:55:48.824294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 06:55:48.824759 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 06:55:48.825024 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:48.828368 (+ 3344us) server_negotiation.cc:299] Negotiation successful
0810 06:55:48.828685 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 06:55:48.837954 30184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.838312 30184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:48.840790 30214 catalog_manager.cc:1347] Generated new cluster ID: ef2f3ebe43d449c1acbb250ebc8a9fbb
I20260810 06:55:48.840896 30214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 06:55:48.847368 30018 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.847870 30018 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52dd6888e1524e6e8715d037ab803d1e, State: Initialized, Role: FOLLOWER
I20260810 06:55:48.848428 30018 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } }
I20260810 06:55:48.855460 30237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.857672 30237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:48.857547 30018 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 06:55:48.857627 29975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "52dd6888e1524e6e8715d037ab803d1e"
I20260810 06:55:48.858294 29974 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 06:55:48.862299 29974 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [term 1 FOLLOWER]: Refusing update from remote peer 6eb06327f7a146c1bc305c2eac1e3a8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 06:55:48.865835 29975 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb06327f7a146c1bc305c2eac1e3a8f in term 1.
I20260810 06:55:48.871694 30196 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 06:55:48.881273 30214 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 06:55:48.883272 30214 catalog_manager.cc:1504] Loading token signing keys...
I20260810 06:55:48.911676 30184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.911993 30184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f [sys.catalog]: This master's current role is: LEADER
I20260810 06:55:48.912729 30237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eb06327f7a146c1bc305c2eac1e3a8f. Latest consensus state: current_term: 1 leader_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.913045 30237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:48.913352 30245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb06327f7a146c1bc305c2eac1e3a8f" member_type: VOTER last_known_addr { host: "127.22.11.190" port: 40563 } } peers { permanent_uuid: "52dd6888e1524e6e8715d037ab803d1e" member_type: VOTER last_known_addr { host: "127.22.11.189" port: 37235 } } peers { permanent_uuid: "47ee2f59b265453f91581709d77642eb" member_type: VOTER last_known_addr { host: "127.22.11.188" port: 44099 } } }
I20260810 06:55:48.913779 30245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 06:55:48.933568 30214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eb06327f7a146c1bc305c2eac1e3a8f: Generated new TSK 0
I20260810 06:55:48.934652 30214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 06:55:48.961515 30277 catalog_manager.cc:1259] Loaded cluster ID: ef2f3ebe43d449c1acbb250ebc8a9fbb
I20260810 06:55:48.961623 30277 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e: loading cluster ID for follower catalog manager: success
I20260810 06:55:48.965432 30277 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e: acquiring CA information for follower catalog manager: success
I20260810 06:55:48.967751 30277 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 52dd6888e1524e6e8715d037ab803d1e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 06:55:49.098119 30020 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.063264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.22.11.190:40563)
0810 06:55:49.063447 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.063454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.093468 (+ 30014us) server_negotiation.cc:407] Connection header received
0810 06:55:49.096283 (+ 2815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.096290 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.096505 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.096598 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.097855 (+ 1257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39040: BlockingRecv error: recv got EOF from 127.0.0.1:39040 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260810 06:55:49.100314 30026 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.064924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.22.11.189:37235)
0810 06:55:49.065104 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.065109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.095953 (+ 30844us) server_negotiation.cc:407] Connection header received
0810 06:55:49.098809 (+ 2856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.098815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.098918 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.098993 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.100078 (+ 1085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60120: BlockingRecv error: recv got EOF from 127.0.0.1:60120 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 06:55:49.103282 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.074844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.22.11.188:44099)
0810 06:55:49.075019 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.075024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.101138 (+ 26114us) server_negotiation.cc:407] Connection header received
0810 06:55:49.101404 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.101410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.101501 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.101582 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.102991 (+ 1409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33986: BlockingRecv error: recv got EOF from 127.0.0.1:33986 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:55:49.153918 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.134666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.22.11.189:37235)
0810 06:55:49.134861 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.134868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.145478 (+ 10610us) server_negotiation.cc:407] Connection header received
0810 06:55:49.146110 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.146117 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.146412 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.146568 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.147490 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.148646 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.150674 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.151365 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.151476 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.152284 (+ 808us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.153037 (+ 753us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.153171 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.153438 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.153537 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 06:55:49.200562 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.187587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33992 (local address 127.22.11.188:44099)
0810 06:55:49.187798 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.187805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.198354 (+ 10549us) server_negotiation.cc:407] Connection header received
0810 06:55:49.198930 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.198937 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.199083 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.199175 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.200319 (+ 1144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33992: BlockingRecv error: recv got EOF from 127.0.0.1:33992 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 06:55:49.218176 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.210989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.22.11.189:37235)
0810 06:55:49.211184 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.211191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.211790 (+ 599us) server_negotiation.cc:407] Connection header received
0810 06:55:49.212004 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.212011 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.212125 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.212242 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.212803 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.214022 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.215995 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.216750 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.216866 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.217358 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.217514 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.217607 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.217765 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.217843 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 06:55:49.227782 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.225550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33998 (local address 127.22.11.188:44099)
0810 06:55:49.225726 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.225731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.226256 (+ 525us) server_negotiation.cc:407] Connection header received
0810 06:55:49.226446 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.226452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.226560 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.226672 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.227547 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33998: BlockingRecv error: recv got EOF from 127.0.0.1:33998 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:55:49.240118 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.233432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.22.11.189:37235)
0810 06:55:49.233607 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.233613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.234131 (+ 518us) server_negotiation.cc:407] Connection header received
0810 06:55:49.234319 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.234325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.234407 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.234504 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.234946 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.236118 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.238027 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.238685 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.238806 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.239316 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.239446 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.239540 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.239709 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.239797 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 06:55:49.254748 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.247306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.22.11.188:44099)
0810 06:55:49.247495 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.247501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.253391 (+ 5890us) server_negotiation.cc:407] Connection header received
0810 06:55:49.253559 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.253565 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.253656 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.253755 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.254526 (+ 771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34008: BlockingRecv error: recv got EOF from 127.0.0.1:34008 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:55:49.268577 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.261784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.22.11.189:37235)
0810 06:55:49.262020 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.262026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.262562 (+ 536us) server_negotiation.cc:407] Connection header received
0810 06:55:49.262772 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.262777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.262928 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.263025 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.263489 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.264602 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.266530 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.267232 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.267351 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.267844 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.267988 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.268076 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.268216 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.268304 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
W20260810 06:55:49.277781 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.275627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.22.11.188:44099)
0810 06:55:49.275878 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.275884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.276358 (+ 474us) server_negotiation.cc:407] Connection header received
0810 06:55:49.276519 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.276525 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.276638 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.276739 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.277545 (+ 806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34018: BlockingRecv error: recv got EOF from 127.0.0.1:34018 (error 108)
Metrics: {"server-negotiator.queue_time_us":143}
I20260810 06:55:49.295712 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.288903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.22.11.189:37235)
0810 06:55:49.289077 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.289083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.289606 (+ 523us) server_negotiation.cc:407] Connection header received
0810 06:55:49.289823 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.289829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.289929 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.290027 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.290515 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.291663 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.293594 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.294249 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.294376 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.294828 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.295037 (+ 209us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.295126 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.295288 (+ 162us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.295371 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 06:55:49.306504 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.302872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34030 (local address 127.22.11.188:44099)
0810 06:55:49.303034 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.303040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.305072 (+ 2032us) server_negotiation.cc:407] Connection header received
0810 06:55:49.305249 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.305255 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.305352 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.305455 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.306289 (+ 834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34030: BlockingRecv error: recv got EOF from 127.0.0.1:34030 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 06:55:49.330900 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.325313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60174 (local address 127.22.11.189:37235)
0810 06:55:49.325508 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.325514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.325748 (+ 234us) server_negotiation.cc:407] Connection header received
0810 06:55:49.325946 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.325952 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.326038 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.326145 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.326522 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.327695 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.329051 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.329706 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.329843 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.330182 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.330322 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.330415 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.330510 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.330597 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 06:55:49.339792 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.337946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.22.11.188:44099)
0810 06:55:49.338134 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.338139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.338459 (+ 320us) server_negotiation.cc:407] Connection header received
0810 06:55:49.338654 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.338660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.338755 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.338880 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.339558 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34040: BlockingRecv error: recv got EOF from 127.0.0.1:34040 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 06:55:49.381080 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.375219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.22.11.189:37235)
0810 06:55:49.375458 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.375465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.375824 (+ 359us) server_negotiation.cc:407] Connection header received
0810 06:55:49.375995 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.376002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.376113 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.376226 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.376578 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.377726 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.379126 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.379822 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.379973 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.380315 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.380452 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.380562 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.380657 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.380741 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 06:55:49.388546 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.386725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.22.11.188:44099)
0810 06:55:49.386948 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.386954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.387307 (+ 353us) server_negotiation.cc:407] Connection header received
0810 06:55:49.387452 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.387458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.387558 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.387675 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.388344 (+ 669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34056: BlockingRecv error: recv got EOF from 127.0.0.1:34056 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 06:55:49.460322 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.455751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.22.11.189:37235)
0810 06:55:49.455969 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.455975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.456225 (+ 250us) server_negotiation.cc:407] Connection header received
0810 06:55:49.456402 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.456406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.456478 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.456553 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.456915 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.457612 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.458898 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.459332 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.459449 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.459752 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.459851 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.459911 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.459997 (+ 86us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.460057 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 06:55:49.467216 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.465455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34062 (local address 127.22.11.188:44099)
0810 06:55:49.465665 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.465671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.465921 (+ 250us) server_negotiation.cc:407] Connection header received
0810 06:55:49.466131 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.466135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.466209 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.466295 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.467022 (+ 727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34062: BlockingRecv error: recv got EOF from 127.0.0.1:34062 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 06:55:49.603289 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.598657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.22.11.189:37235)
0810 06:55:49.598810 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.598814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.599195 (+ 381us) server_negotiation.cc:407] Connection header received
0810 06:55:49.599386 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.599390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.599460 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.599527 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.599871 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.600569 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.601970 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.602369 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.602459 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.602779 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.602887 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.602951 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.603026 (+ 75us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.603087 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260810 06:55:49.609862 30160 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 06:55:49.608290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.22.11.188:44099)
0810 06:55:49.608493 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.608498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.608752 (+ 254us) server_negotiation.cc:407] Connection header received
0810 06:55:49.608889 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.608893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.608975 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.609065 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 06:55:49.609669 (+ 604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34072: BlockingRecv error: recv got EOF from 127.0.0.1:34072 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 06:55:49.649632 30180 catalog_manager.cc:1259] Loaded cluster ID: ef2f3ebe43d449c1acbb250ebc8a9fbb
I20260810 06:55:49.649725 30180 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: loading cluster ID for follower catalog manager: success
I20260810 06:55:49.652194 30180 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: acquiring CA information for follower catalog manager: success
I20260810 06:55:49.653977 30180 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 47ee2f59b265453f91581709d77642eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 06:55:49.874406 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.869479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.22.11.189:37235)
0810 06:55:49.869713 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.869719 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.869878 (+ 159us) server_negotiation.cc:407] Connection header received
0810 06:55:49.870098 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.870102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.870174 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.870254 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.870676 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.871406 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.872750 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.873204 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.873316 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.873750 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.873859 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.873932 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.874085 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.874166 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 06:55:49.885082 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.879772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.22.11.188:44099)
0810 06:55:49.879964 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.879970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.880100 (+ 130us) server_negotiation.cc:407] Connection header received
0810 06:55:49.880276 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.880280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.880454 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.880561 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.880992 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.881676 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.882963 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.883341 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.883446 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.883909 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.884460 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.884608 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.884724 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.884805 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 06:55:49.900524 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.890918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34102 (local address 127.22.11.188:44099)
0810 06:55:49.891109 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.891115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.894773 (+ 3658us) server_negotiation.cc:407] Connection header received
0810 06:55:49.894918 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.894923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.895016 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.895104 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.895899 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.896843 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.898624 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.899178 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.899282 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.899775 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.899898 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.899982 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.900143 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.900232 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 06:55:49.901391 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.890508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39044 (local address 127.22.11.190:40563)
0810 06:55:49.894390 (+ 3882us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.894397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.894418 (+ 21us) server_negotiation.cc:407] Connection header received
0810 06:55:49.894546 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.894552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.894793 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.894968 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.895532 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.896511 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.898390 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.898910 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.899034 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.899596 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.900004 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.900124 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.901000 (+ 876us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.901097 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3766,"thread_start_us":3671,"threads_started":1}
I20260810 06:55:49.902024 30026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 06:55:49.890532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.22.11.189:37235)
0810 06:55:49.890753 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 06:55:49.890759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 06:55:49.895392 (+ 4633us) server_negotiation.cc:407] Connection header received
0810 06:55:49.895827 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 06:55:49.895833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 06:55:49.895916 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 06:55:49.895962 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 06:55:49.896523 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.897594 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.899333 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 06:55:49.899814 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 06:55:49.899908 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 06:55:49.900528 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 06:55:49.900693 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 06:55:49.900776 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 06:55:49.901675 (+ 899us) server_negotiation.cc:299] Negotiation successful
0810 06:55:49.901758 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 06:55:49.906189 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29776
I20260810 06:55:49.907094 29881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:50.016883 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29776
W20260810 06:55:50.046113 30068 connection.cc:535] server connection from 127.0.0.1:33958 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 06:55:50.046553 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 29905
I20260810 06:55:50.047561 30010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:50.153941 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 29905
I20260810 06:55:50.187044 22574 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu pid 30038
I20260810 06:55:50.188181 30139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 06:55:50.294327 22574 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskJIbnRf/build/asan/bin/kudu with pid 30038
2026-08-10T06:55:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4758 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9540 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (370142 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST